-- Logs begin at Mon 2022-10-17 03:01:44 UTC, end at Mon 2022-10-17 04:15:05 UTC. -- Oct 17 03:31:30.774710 dv-cinder-815582-16-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 17 03:31:30.784623 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 17 03:31:30.786636 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Oct 17 03:31:30 2022] *** Oct 17 03:31:30.786636 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 17 03:31:30.786636 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 17 03:31:30.786636 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: nodename: dv-cinder-815582-16-iscsi Oct 17 03:31:30.786636 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: machine: x86_64 Oct 17 03:31:30.786636 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: clock source: unix Oct 17 03:31:30.786636 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: pcre jit disabled Oct 17 03:31:30.786636 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: detected number of CPU cores: 4 Oct 17 03:31:30.786636 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: current working directory: / Oct 17 03:31:30.786636 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: detected binary path: /usr/bin/uwsgi-core Oct 17 03:31:30.787194 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: your processes number limit is 47680 Oct 17 03:31:30.787194 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: your memory page size is 4096 bytes Oct 17 03:31:30.787194 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: detected max file descriptor number: 2048 Oct 17 03:31:30.787194 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: lock engine: pthread robust mutexes Oct 17 03:31:30.787194 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: thunder lock: enabled Oct 17 03:31:30.787194 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 17 03:31:30.787194 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 17 03:31:30.825564 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: Python main interpreter initialized at 0x5558e01009c0 Oct 17 03:31:30.825564 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: python threads support enabled Oct 17 03:31:30.825564 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: your server socket listen backlog is limited to 100 connections Oct 17 03:31:30.825564 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: your mercy for graceful operations on workers is 90 seconds Oct 17 03:31:30.825877 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: mapped 403077 bytes (393 KB) for 2 cores Oct 17 03:31:30.825981 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: *** Operational MODE: preforking *** Oct 17 03:31:30.826252 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: *** uWSGI is running in multiple interpreter mode *** Oct 17 03:31:30.826252 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: spawned uWSGI master process (pid: 88287) Oct 17 03:31:30.826443 dv-cinder-815582-16-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 17 03:31:30.827253 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: spawned uWSGI worker 1 (pid: 88299, cores: 1) Oct 17 03:31:30.827343 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: spawned uWSGI worker 2 (pid: 88300, cores: 1) Oct 17 03:31:30.829042 dv-cinder-815582-16-iscsi devstack@keystone.service[88287]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 17 03:31:35.159211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 17 03:31:35.159211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: from cryptography.utils import int_from_bytes Oct 17 03:31:35.160124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 17 03:31:35.160124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: from cryptography.utils import int_from_bytes Oct 17 03:31:35.160664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 17 03:31:35.160664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: from cryptography.utils import int_from_bytes Oct 17 03:31:35.161127 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 17 03:31:35.161127 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: from cryptography.utils import int_from_bytes Oct 17 03:31:35.209036 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 17 03:31:35.213720 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 17 03:31:35.213720 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] command line args: [] {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 17 03:31:35.213720 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 17 03:31:35.213720 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ================================================================================ {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 17 03:31:35.213720 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] admin_token = **** {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.213720 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] config_dir = [] {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.214130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 17 03:31:35.214199 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.214199 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] config_source = [] {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.214316 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 17 03:31:35.214382 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] debug = True {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.214447 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] command line args: [] {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 17 03:31:35.214505 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.214892 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 17 03:31:35.214892 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ================================================================================ {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 17 03:31:35.215022 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.215086 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] admin_token = **** {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.215150 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.215214 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] config_dir = [] {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.215273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.215338 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.215402 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.215466 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] config_source = [] {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.215525 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] list_limit = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.215587 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] debug = True {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.215656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] log_config_append = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.215715 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.216007 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.216073 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.216137 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] log_dir = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.216207 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.216276 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] log_file = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.216341 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.216405 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.216724 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.216786 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.216849 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] list_limit = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.216914 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.216978 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] log_config_append = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.217036 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.217095 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.217158 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.217225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] log_dir = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.217290 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.217350 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] log_file = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.217407 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.217463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.217525 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.217595 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.217654 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.217711 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.217774 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.217832 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.217896 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.217965 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.218030 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.218094 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.218164 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.218224 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.218292 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.218364 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.218427 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.218486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.218486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.218594 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.219167 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.219232 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] publish_errors = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.219295 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.219353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.219417 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.219476 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.219538 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.219596 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.219652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.219708 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.219708 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.219708 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] standard_threads = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.219900 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.219900 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] publish_errors = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.220072 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.220151 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.220211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.220211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.220323 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.220391 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] use_journal = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.220450 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.220507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] use_json = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.220568 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.220626 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] use_stderr = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.220682 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.220748 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] use_syslog = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.220748 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.220874 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] standard_threads = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.220874 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.220986 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.220986 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.221096 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.221096 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.221206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.221264 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] use_journal = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.221326 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.221394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] use_json = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.221485 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] use_stderr = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.221585 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.221653 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] use_syslog = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.221746 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.221881 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 17 03:31:35.222026 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.222136 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.222214 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.222396 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.222491 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.222636 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.222725 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.222875 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.222954 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.223142 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.223223 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.223362 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.223449 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.223594 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.223672 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.223859 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.223940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.224107 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.224196 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.224342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.224421 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.224593 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.224676 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.224830 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.224920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.225072 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.225150 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.225307 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.225387 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.225553 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.225642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.225803 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.225890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.226041 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.226120 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.226287 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.226458 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.226540 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.226619 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.226731 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.226852 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.226983 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.227073 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.227179 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.227323 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.227423 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.227586 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.227666 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.227868 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.227958 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.228119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.228198 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.228352 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.228439 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.228584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.228663 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.228816 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.228894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.229052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.229147 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.229283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.229361 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.229518 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.229598 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.229751 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.229865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.229971 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.230069 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.230193 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.230299 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.230419 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.230536 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.230636 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.230752 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.230870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.230955 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.231114 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.231242 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.231342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.231427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.231574 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.231652 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.231826 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.231998 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.232115 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.232214 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.232293 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.232354 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.232501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.232617 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.232773 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.232874 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.232947 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.233009 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.233173 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.233306 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.233413 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.233488 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.233625 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.233719 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.233857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.233935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.234084 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.234175 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.234381 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.235071 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.235156 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.235276 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.235362 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.235437 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.235554 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.235644 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.235826 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.235941 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.236047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.236126 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.236277 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.236357 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.236506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.236586 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.236737 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.236818 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.236979 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.237055 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.237215 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.237299 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.237448 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.237535 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.237688 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.237772 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.237926 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.238011 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.238156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.238241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.238387 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.238466 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.238621 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.238698 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.238852 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.238936 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.239123 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.239216 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.239328 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.239408 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.239559 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.239636 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.239794 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.239879 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.240028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.240108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.243619 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.243766 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.243846 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.243930 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.244010 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.244089 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.244169 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.244251 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.244335 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.244409 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.244481 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.244553 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.244624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.244696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.244778 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.244852 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.244929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.245002 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.245075 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.245146 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.245225 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.245298 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.245370 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.245442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.245513 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.245585 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.245663 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.245742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.245815 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.245887 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.245963 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.246036 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.246118 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.246192 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.246263 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.246335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.246414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.246488 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.246569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.246643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.246714 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.246786 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.246863 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.246942 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.247023 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.247096 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.247168 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.247243 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.247315 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.247386 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.247465 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] auth.external = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.247539 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.247611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.247722 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.247839 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.247925 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.248000 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.248099 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.248184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] auth.password = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.248258 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.248331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] auth.token = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.248403 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.248475 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.248547 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.248627 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.248627 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.248767 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.248844 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.248917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] auth.external = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.249000 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.249082 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.249156 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.249228 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.249301 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.249373 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.249453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] credential.caching = True {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.249527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] auth.password = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.249600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.249672 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] auth.token = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.249748 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.249822 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.249925 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.250004 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.250077 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.250156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.250229 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.250302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.250382 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.250456 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.250529 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.250606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.250679 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.250752 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.250902 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.250986 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] credential.caching = True {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.251154 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.251234 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.251436 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.251515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.251677 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.251778 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.251946 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.252028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.252190 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.252270 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.252427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.252508 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.252661 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] federation.caching = True {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.252740 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.252897 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.252988 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.253139 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.253214 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.253371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.253450 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.253599 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.253758 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.253858 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.253934 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.254061 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.254201 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.254311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.254445 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.254533 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.254702 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.254782 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.254944 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] federation.caching = True {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.255033 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.255184 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.255273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.255431 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.255515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] identity.caching = True {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.255677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.255787 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.255945 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.256037 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.256182 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.256264 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.256415 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.256496 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.256660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.256756 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.256890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.256970 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.257158 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.257248 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.257370 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.257451 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.257617 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.257697 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.257847 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.257927 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] identity.caching = True {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.258082 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.258163 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.258308 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.258389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.258550 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.258635 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.258788 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.258869 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.259029 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.259109 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.259296 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.259383 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.259520 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.259604 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.259784 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.259870 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.260024 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.260105 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.260259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.260340 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.260487 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.260568 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.260720 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.260799 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.260950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.261038 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.261184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.261264 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.261425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.261506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.261665 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.261751 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.261912 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.261991 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.262139 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.262224 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.262386 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.262467 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.262628 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.262707 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.262859 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.262940 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.263094 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.263178 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.263325 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.263404 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.263551 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.263630 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.263808 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.263892 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.264037 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.264117 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.264271 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.264350 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.264505 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.264586 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.264740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.264819 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.264973 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.265053 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.265213 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.265319 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.265459 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.266081 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.266266 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.266380 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.266488 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.266626 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.266730 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.266861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.266962 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.267112 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.267200 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.267358 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.267439 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.267593 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.267673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.267868 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.267962 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.268106 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.268241 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.268322 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.user = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.268499 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.268579 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.268754 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.268836 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.268988 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.269068 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.269220 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.269300 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.269469 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.269550 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.269724 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.269817 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.269958 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.270048 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.270193 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.270285 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.270427 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.270508 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.270656 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.user = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.270743 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.270889 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.270968 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.271118 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.271198 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.271363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.271452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.271609 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.271688 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.271878 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.271960 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.272122 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.272308 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.272430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.272510 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.272584 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.272655 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.272816 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.272905 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.273050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.273130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.273284 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.273363 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.273517 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.273606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.273775 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.273863 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.274017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.274097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.274263 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.274355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.274495 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.274574 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.274743 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.274822 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.274971 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.275057 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.275213 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.275298 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.275447 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.275527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.275680 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.275864 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.275945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.276018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.276169 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.276250 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.276413 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.276568 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.276645 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.276718 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.276863 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.276943 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.277102 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] resource.caching = True {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.277190 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.277346 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.277424 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.277588 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.277667 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.277825 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.277925 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.278068 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.278149 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.278315 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.278395 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.278554 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.278642 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.278792 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.278871 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.279024 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.279103 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.279261 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.279352 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.279491 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] role.caching = True {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.279570 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] resource.caching = True {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.279779 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] role.driver = sql {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.279864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.280006 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.280098 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.280244 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.280326 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.280495 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.280575 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.280747 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.280835 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.280984 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.281064 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.281770 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.281863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.282034 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.282123 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.282284 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.282365 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.282533 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.282613 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] role.caching = True {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.282765 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.282853 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] role.driver = sql {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.283010 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.283090 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.283258 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.283336 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.283500 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.283580 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.283761 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.283850 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.284083 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.284162 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.284337 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.284420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.284588 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.284668 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.284838 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.284918 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.285089 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.285171 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.285328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.285407 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.285562 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.285642 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.285815 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.285894 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.286062 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.286142 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.286296 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.286375 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.286528 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.286701 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.286780 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.286940 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.287023 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.287192 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.287280 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.287445 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.287524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.287691 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.287801 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.287951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.288043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.288184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.288264 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.288418 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.288497 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.288651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] token.caching = True {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.288739 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.288883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.288971 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.289326 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.289406 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.289490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.289554 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.289614 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.289672 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.289824 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.289914 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.290058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.290140 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.290298 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.290378 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.290540 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.290620 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.290785 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.290867 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.291018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.291098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.291255 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] token.caching = True {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.291376 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.291469 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.291533 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.291713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.291854 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.291976 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.292054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.292251 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.292347 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.292468 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.292549 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.292706 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.292786 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.292952 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.293032 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] database.connection = **** {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.293198 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.293278 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.293446 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.293526 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.293683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.293763 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.293922 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.294004 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.294155 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.294234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.294387 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.294465 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.294619 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.294700 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.294850 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.294931 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.295093 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.295181 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.295328 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.295416 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.295568 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.295766 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] database.connection = **** {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.295905 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.296020 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.296099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.301024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.301109 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.301186 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.301248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.301332 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.301408 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.301491 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.301567 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.301640 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.301713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.301786 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.301859 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88299) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 17 03:31:35.301940 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.301940 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.301940 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.301940 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.301940 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.301940 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.301940 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.301940 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.301940 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.302481 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.302481 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 17 03:31:35.302481 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88300) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 17 03:31:35.326259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.326484 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.326711 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.328004 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.328752 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.329893 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.335494 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.337389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.363899 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.364125 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.366060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.366292 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.421000 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.421141 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.468535 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.474467 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.474563 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.474563 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.637605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.637822 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.637914 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.638009 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.638077 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.638153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.638289 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.638374 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.638506 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.638591 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.638720 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.638802 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.661912 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.662143 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.662230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.662371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.662455 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.662597 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.662680 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.662817 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.662901 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.663034 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.663117 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.663270 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.663369 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.663474 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.674329 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.674575 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.686764 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.686994 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.687213 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.687300 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.687974 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.688060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.688196 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.688280 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.688409 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.688492 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.688622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88299) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.688714 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.688837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.689037 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88300) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 17 03:31:35.708514 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.708728 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 17 03:31:35.708831 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.708942 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 17 03:31:35.709072 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.709272 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 17 03:31:35.709354 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.709479 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 17 03:31:35.709569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.709753 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.709835 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.710185 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.710280 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.710280 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.710434 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.710509 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.720146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.720379 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.720464 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.720664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.720750 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.720940 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.721023 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.721228 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.727868 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.728199 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.730778 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.731028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.731110 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.731310 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.731393 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.731595 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.731677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.731906 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.731990 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.732171 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.732256 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.732440 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.732522 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.732714 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.732802 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.732983 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.733063 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.733255 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.733334 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.733523 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.733604 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.733794 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.756567 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.756892 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.758021 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.759497 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.759581 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.759581 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.762998 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.763283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.763380 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.763598 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.763690 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.763900 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.770548 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.770804 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.770890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.771093 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.776444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.776742 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.776865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.777034 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.777117 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.777324 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.777407 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.777605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.777696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.777885 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.777968 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.778162 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.778244 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.778441 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.778524 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.778722 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.793835 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.794114 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.794198 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.799547 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.799680 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.799774 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.801147 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.801447 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.801608 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.801892 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.805683 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.806144 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.808746 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.809372 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.811935 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.812234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.813033 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.813149 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.813316 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.813450 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.813593 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.813740 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.813869 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.814030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.814311 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.814601 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.830100 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.830388 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.830660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.830925 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.831007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.831265 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.831538 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.831834 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.837777 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.838662 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.839297 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.840220 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.840854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.841132 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.842020 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.842301 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.844135 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.844419 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.844702 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.844996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.845104 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.845393 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.845476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.845680 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.845978 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.846371 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.856520 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.856803 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.857086 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.857397 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.857517 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.857763 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.857845 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.858055 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.858151 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.858335 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.858418 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.858730 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.859019 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.859335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.873673 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.874657 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.877185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.878106 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.880126 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.881032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.881600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.882489 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.883074 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.883385 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.883783 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.883992 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.884279 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.884697 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.890138 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.891585 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.892220 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.893094 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.895120 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.895517 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.895826 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.895997 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.896104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.896398 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.896690 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.896969 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.903887 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.904176 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.904461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.904759 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.905055 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.905344 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.911324 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88299) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.911609 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.911915 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.912205 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.912287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88300) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 17 03:31:35.912479 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.912561 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.912751 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.912834 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.913024 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.913108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.913303 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.913384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.913574 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.913664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.913861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.913942 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.914148 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.914228 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.914408 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.914490 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.914684 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.914766 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.914953 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88299) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.915043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.915218 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.915487 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.915809 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88300) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 17 03:31:35.949192 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5558e01009c0 pid: 88299 (default app) Oct 17 03:31:35.950040 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5558e01009c0 pid: 88300 (default app) Oct 17 03:31:35.952454 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132b622-d89c-490a-9f13-bf85d20a45f9 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:35.952738 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132b622-d89c-490a-9f13-bf85d20a45f9 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:35.953602 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132b622-d89c-490a-9f13-bf85d20a45f9 None None] PATH_INFO: `/v3` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:35.957777 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Mon Oct 17 03:31:35 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:31:43.540304 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165368de-40ed-4d18-8c9e-e239975330d0 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:43.540842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165368de-40ed-4d18-8c9e-e239975330d0 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:43.540842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165368de-40ed-4d18-8c9e-e239975330d0 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:43.545102 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1/2] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:31:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:31:43.550810 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58c74c-4812-4acb-a6ea-a60182536721 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:43.551247 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58c74c-4812-4acb-a6ea-a60182536721 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:43.551342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58c74c-4812-4acb-a6ea-a60182536721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:43.621360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: INFO dbcounter [None req-9e58c74c-4812-4acb-a6ea-a60182536721 None None] Registered counter for database keystone Oct 17 03:31:43.640695 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG oslo_db.sqlalchemy.engines [None req-9e58c74c-4812-4acb-a6ea-a60182536721 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88299) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 17 03:31:43.646967 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writer thread running {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 17 03:31:43.827414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG passlib.handlers.bcrypt [None req-9e58c74c-4812-4acb-a6ea-a60182536721 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=88299) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 17 03:31:43.827838 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG passlib.handlers.bcrypt [None req-9e58c74c-4812-4acb-a6ea-a60182536721 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88299) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 17 03:31:43.886430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-9e58c74c-4812-4acb-a6ea-a60182536721 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:31:43.888059 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-9e58c74c-4812-4acb-a6ea-a60182536721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:31:44.014090 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 2/3] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:31:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 03:31:44.487531 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165368de-40ed-4d18-8c9e-e239975330d0 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:31:44.556489 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: INFO dbcounter [None req-165368de-40ed-4d18-8c9e-e239975330d0 None None] Registered counter for database keystone Oct 17 03:31:44.577205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG oslo_db.sqlalchemy.engines [None req-165368de-40ed-4d18-8c9e-e239975330d0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88300) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 17 03:31:44.583880 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writer thread running {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 17 03:31:44.700218 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165368de-40ed-4d18-8c9e-e239975330d0 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:31:44.705839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a89f0-d2d7-4181-be47-b30578f2696a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:31:44.706578 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a89f0-d2d7-4181-be47-b30578f2696a admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:44.706863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a89f0-d2d7-4181-be47-b30578f2696a admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:44.707132 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a89f0-d2d7-4181-be47-b30578f2696a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:44.715542 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a89f0-d2d7-4181-be47-b30578f2696a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:31:44.766838 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f70a89f0-d2d7-4181-be47-b30578f2696a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 17 03:31:44.767126 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a89f0-d2d7-4181-be47-b30578f2696a admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:31:44.776248 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 2/4] 10.222.0.17 () {58 vars in 1324 bytes} [Mon Oct 17 03:31:44 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 17 03:31:44.782257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58c74c-4812-4acb-a6ea-a60182536721 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:31:44.798987 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58c74c-4812-4acb-a6ea-a60182536721 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:31:44.803355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641f1e5-0e75-4fa7-9d35-828480b77bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:31:44.804491 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641f1e5-0e75-4fa7-9d35-828480b77bbd admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:44.804769 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641f1e5-0e75-4fa7-9d35-828480b77bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:44.805036 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641f1e5-0e75-4fa7-9d35-828480b77bbd admin admin] PATH_INFO: `/v3/projects/6f518d7d57a2406a8345926faf8a2f14` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:44.806603 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641f1e5-0e75-4fa7-9d35-828480b77bbd admin admin] RBAC: Authorizing `identity:get_project(project_id=6f518d7d57a2406a8345926faf8a2f14)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:31:44.858987 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5641f1e5-0e75-4fa7-9d35-828480b77bbd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 17 03:31:44.859338 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641f1e5-0e75-4fa7-9d35-828480b77bbd admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:31:44.863886 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 3/5] 10.222.0.17 () {56 vars in 1207 bytes} [Mon Oct 17 03:31:44 2022] GET /identity/v3/projects/6f518d7d57a2406a8345926faf8a2f14 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:31:46.978013 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6f88d-a205-457b-9ca1-5a4c8dbe38cc None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:46.978529 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6f88d-a205-457b-9ca1-5a4c8dbe38cc None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:46.978622 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6f88d-a205-457b-9ca1-5a4c8dbe38cc None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:46.982688 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 3/6] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:31:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:31:46.988240 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec73a51-2129-4026-ae43-5f4fe04269bb None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:46.988646 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec73a51-2129-4026-ae43-5f4fe04269bb None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:46.988809 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec73a51-2129-4026-ae43-5f4fe04269bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:47.009163 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-aec73a51-2129-4026-ae43-5f4fe04269bb None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:31:47.010671 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-aec73a51-2129-4026-ae43-5f4fe04269bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:31:47.019099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 4/7] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:31:46 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 03:31:47.286606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6f88d-a205-457b-9ca1-5a4c8dbe38cc None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:31:47.299038 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6f88d-a205-457b-9ca1-5a4c8dbe38cc None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:31:47.303225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cdf33-6cc6-4b2b-ba41-b9a8b6d2b07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:31:47.303852 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cdf33-6cc6-4b2b-ba41-b9a8b6d2b07d admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:47.304135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cdf33-6cc6-4b2b-ba41-b9a8b6d2b07d admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:47.304405 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cdf33-6cc6-4b2b-ba41-b9a8b6d2b07d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:47.312526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8cdf33-6cc6-4b2b-ba41-b9a8b6d2b07d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:31:47.314268 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8cdf33-6cc6-4b2b-ba41-b9a8b6d2b07d admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:31:47.322903 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 4/8] 10.222.0.17 () {58 vars in 1324 bytes} [Mon Oct 17 03:31:47 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 03:31:47.328075 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec73a51-2129-4026-ae43-5f4fe04269bb None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:31:47.337062 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec73a51-2129-4026-ae43-5f4fe04269bb None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:31:47.341689 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7eb157-8adb-4fc4-a5be-218577b22c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:31:47.342322 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7eb157-8adb-4fc4-a5be-218577b22c3c admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:47.342607 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7eb157-8adb-4fc4-a5be-218577b22c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:47.342883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7eb157-8adb-4fc4-a5be-218577b22c3c admin admin] PATH_INFO: `/v3/users/f09f98306cd54b068020428d74bd0957` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:47.344257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7eb157-8adb-4fc4-a5be-218577b22c3c admin admin] RBAC: Authorizing `identity:get_user(user_id=f09f98306cd54b068020428d74bd0957)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:31:47.346076 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7eb157-8adb-4fc4-a5be-218577b22c3c admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:31:47.347589 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 5/9] 10.222.0.17 () {56 vars in 1198 bytes} [Mon Oct 17 03:31:47 2022] GET /identity/v3/users/f09f98306cd54b068020428d74bd0957 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:31:49.504305 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491209a2-c87c-42a3-a145-84eeed4391a5 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:49.504819 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491209a2-c87c-42a3-a145-84eeed4391a5 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:49.504927 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491209a2-c87c-42a3-a145-84eeed4391a5 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:49.506174 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 5/10] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:31:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:31:49.511326 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4918f5-ce71-497e-9efe-d1fd6b73ac36 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:49.511757 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4918f5-ce71-497e-9efe-d1fd6b73ac36 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:49.512009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4918f5-ce71-497e-9efe-d1fd6b73ac36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:49.529566 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-fe4918f5-ce71-497e-9efe-d1fd6b73ac36 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:31:49.530765 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-fe4918f5-ce71-497e-9efe-d1fd6b73ac36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:31:49.570784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40b6ef-d44b-4ccd-ac54-1cfcb58848d8 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:49.571068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40b6ef-d44b-4ccd-ac54-1cfcb58848d8 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:49.571346 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc40b6ef-d44b-4ccd-ac54-1cfcb58848d8 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:49.572699 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 6/11] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:31:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:31:49.577908 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a5786-9590-47be-b2c9-5ed4b9cc17a9 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:49.578190 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a5786-9590-47be-b2c9-5ed4b9cc17a9 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:49.578463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068a5786-9590-47be-b2c9-5ed4b9cc17a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:49.612089 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 6/12] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:31:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:31:49.620599 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG passlib.handlers.bcrypt [None req-068a5786-9590-47be-b2c9-5ed4b9cc17a9 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=88300) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 17 03:31:49.620990 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG passlib.handlers.bcrypt [None req-068a5786-9590-47be-b2c9-5ed4b9cc17a9 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88300) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 17 03:31:49.669813 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-068a5786-9590-47be-b2c9-5ed4b9cc17a9 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:31:49.671949 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-068a5786-9590-47be-b2c9-5ed4b9cc17a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:31:49.772581 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 7/13] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:31:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 03:31:49.881267 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4918f5-ce71-497e-9efe-d1fd6b73ac36 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:31:49.906467 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:31:49.985191 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4918f5-ce71-497e-9efe-d1fd6b73ac36 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:31:50.017408 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e5a5f-373a-491c-b292-377ce2fbe4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:31:50.018046 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e5a5f-373a-491c-b292-377ce2fbe4e4 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:50.018332 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e5a5f-373a-491c-b292-377ce2fbe4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:50.018611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197e5a5f-373a-491c-b292-377ce2fbe4e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:50.029541 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197e5a5f-373a-491c-b292-377ce2fbe4e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:31:50.035254 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068a5786-9590-47be-b2c9-5ed4b9cc17a9 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:31:50.039050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-197e5a5f-373a-491c-b292-377ce2fbe4e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:31:50.039050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:31:50.039050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:31:50.039050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:31:50.039050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:31:50.039050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:31:50.039050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:31:50.039050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:31:50.039050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:31:50.039050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:31:50.039050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:31:50.039050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:31:50.039050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:31:50.039050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:31:50.039050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:31:50.039050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:31:50.040009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:31:50.040009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:31:50.040009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:31:50.040009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:31:50.040009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:31:50.040009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:31:50.040009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:31:50.040009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:31:50.040009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:31:50.040009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:31:50.040009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:31:50.040009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:31:50.040009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:31:50.040009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:31:50.040009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:31:50.040009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:31:50.040747 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:31:50.040747 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:31:50.040747 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:31:50.040747 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:31:50.040747 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:31:50.040747 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:31:50.040747 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:31:50.040747 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:31:50.040747 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:31:50.040747 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:31:50.040747 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:31:50.040747 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:31:50.040747 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 7/14] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Oct 17 03:31:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:31:50.045852 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e5a5f-373a-491c-b292-377ce2fbe4e4 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:31:50.129571 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:31:50.136188 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068a5786-9590-47be-b2c9-5ed4b9cc17a9 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:31:50.141432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:31:50.147856 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197e5a5f-373a-491c-b292-377ce2fbe4e4 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:31:50.174513 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4176d39-90b2-4944-9353-b163b0d6b3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:31:50.175779 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4176d39-90b2-4944-9353-b163b0d6b3c9 None admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:50.175779 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4176d39-90b2-4944-9353-b163b0d6b3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:50.175975 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4176d39-90b2-4944-9353-b163b0d6b3c9 None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:50.176877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4176d39-90b2-4944-9353-b163b0d6b3c9 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:31:50.178291 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4176d39-90b2-4944-9353-b163b0d6b3c9 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:31:50.183179 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674b941-114e-4213-98f9-2f4ab8ebd345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:31:50.183813 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674b941-114e-4213-98f9-2f4ab8ebd345 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:50.184104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674b941-114e-4213-98f9-2f4ab8ebd345 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:50.184416 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674b941-114e-4213-98f9-2f4ab8ebd345 None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:50.185168 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674b941-114e-4213-98f9-2f4ab8ebd345 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:31:50.185477 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674b941-114e-4213-98f9-2f4ab8ebd345 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:31:50.186896 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674b941-114e-4213-98f9-2f4ab8ebd345 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:31:50.199279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 8/15] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:31:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:31:50.204917 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674b941-114e-4213-98f9-2f4ab8ebd345 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:31:50.232365 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.sql.core [None req-f4176d39-90b2-4944-9353-b163b0d6b3c9 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 03:31:50.232365 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 03:31:50.232365 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [parameters: {'id': 'd4da19c8c71942a5b8db1a970b8ff5b7', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 17 03:31:50.232365 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88300) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 03:31:50.251494 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-f4176d39-90b2-4944-9353-b163b0d6b3c9 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 17 03:31:50.251494 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:31:50.251494 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 03:31:50.251494 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 03:31:50.251494 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 03:31:50.251494 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 03:31:50.251494 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 03:31:50.251494 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 03:31:50.251494 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 03:31:50.251494 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application conn.query(q) Oct 17 03:31:50.251494 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 03:31:50.251494 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 03:31:50.251494 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 03:31:50.251494 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application result.read() Oct 17 03:31:50.252544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 03:31:50.252544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 03:31:50.252544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 03:31:50.252544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 03:31:50.252544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 03:31:50.252544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 03:31:50.252544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 03:31:50.252544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 03:31:50.252544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 03:31:50.252544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:31:50.252544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 03:31:50.252544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:31:50.252544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:31:50.252544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 03:31:50.252544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 03:31:50.252544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 17 03:31:50.252544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 17 03:31:50.252544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 03:31:50.253476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application next(self.gen) Oct 17 03:31:50.253476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 03:31:50.253476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application yield resource Oct 17 03:31:50.253476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 03:31:50.253476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application next(self.gen) Oct 17 03:31:50.253476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 03:31:50.253476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 03:31:50.253476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 03:31:50.253476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 03:31:50.253476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 03:31:50.253476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise self.value Oct 17 03:31:50.253476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 03:31:50.253476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 03:31:50.253476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 03:31:50.253476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application session.commit() Oct 17 03:31:50.253476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 03:31:50.253476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 03:31:50.254484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 03:31:50.254484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 03:31:50.254484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 03:31:50.254484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self.session.flush() Oct 17 03:31:50.254484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 03:31:50.254484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 03:31:50.254484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 03:31:50.254484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 03:31:50.254484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 03:31:50.254484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application compat.raise_( Oct 17 03:31:50.254484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 03:31:50.254484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception Oct 17 03:31:50.254484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 03:31:50.254484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 03:31:50.254484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 03:31:50.254484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 03:31:50.254484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 03:31:50.255368 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 03:31:50.255368 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 03:31:50.255368 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 03:31:50.255368 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 17 03:31:50.255368 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 17 03:31:50.255368 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 03:31:50.255368 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 03:31:50.255368 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 03:31:50.255368 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 03:31:50.255368 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 03:31:50.255368 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 03:31:50.255368 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 03:31:50.255368 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 03:31:50.255368 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 03:31:50.255368 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 03:31:50.256154 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 03:31:50.256154 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception Oct 17 03:31:50.256154 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 03:31:50.256154 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 03:31:50.256154 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 03:31:50.256154 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 03:31:50.256154 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 03:31:50.256154 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 03:31:50.256154 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 03:31:50.256154 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application conn.query(q) Oct 17 03:31:50.256154 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 03:31:50.256154 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 03:31:50.256154 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 03:31:50.256154 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application result.read() Oct 17 03:31:50.256154 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 03:31:50.256154 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 03:31:50.256154 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 03:31:50.256916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 03:31:50.256916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 03:31:50.256916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 03:31:50.256916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 03:31:50.256916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 03:31:50.256916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 03:31:50.256916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 03:31:50.256916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [parameters: {'id': 'd4da19c8c71942a5b8db1a970b8ff5b7', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 17 03:31:50.256916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 03:31:50.256916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:31:50.256916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:31:50.256916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:31:50.256916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:31:50.256916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 17 03:31:50.256916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 17 03:31:50.257677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 03:31:50.257677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 03:31:50.257677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 17 03:31:50.257677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:31:50.257677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:31:50.257677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:31:50.257677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:31:50.257677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:31:50.257677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:31:50.257677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:31:50.257677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:31:50.257677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:31:50.257677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:31:50.257677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:31:50.257677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:31:50.257677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:31:50.257677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:31:50.258463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 17 03:31:50.258463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 17 03:31:50.258463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:31:50.258463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:31:50.258463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 17 03:31:50.258463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 17 03:31:50.258463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:31:50.258463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:31:50.258463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 17 03:31:50.258463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 17 03:31:50.258463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 17 03:31:50.258463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:31:50.258463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 8/16] 10.222.0.17 () {60 vars in 1137 bytes} [Mon Oct 17 03:31:50 2022] POST /identity/v3/domains => generated 190 bytes in 218 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:31:50.258463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4176d39-90b2-4944-9353-b163b0d6b3c9 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:31:50.308653 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674b941-114e-4213-98f9-2f4ab8ebd345 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:31:50.341502 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb66a3-0c44-4fdd-ad99-e919010bfece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:31:50.342139 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb66a3-0c44-4fdd-ad99-e919010bfece None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:50.342424 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb66a3-0c44-4fdd-ad99-e919010bfece None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:50.342696 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb66a3-0c44-4fdd-ad99-e919010bfece None admin] PATH_INFO: `/v3/users/f09f98306cd54b068020428d74bd0957` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:50.344393 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb66a3-0c44-4fdd-ad99-e919010bfece None admin] RBAC: Authorizing `identity:get_user(user_id=f09f98306cd54b068020428d74bd0957)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:31:50.345839 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb66a3-0c44-4fdd-ad99-e919010bfece None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:31:50.347404 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 9/17] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:31:50 2022] GET /identity/v3/users/f09f98306cd54b068020428d74bd0957 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:31:50.353045 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb66a3-0c44-4fdd-ad99-e919010bfece None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:31:50.358514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4176d39-90b2-4944-9353-b163b0d6b3c9 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:31:50.367374 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:31:50.391722 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1c8bb-029b-4fd9-889a-686e071e36ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:31:50.392600 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1c8bb-029b-4fd9-889a-686e071e36ce None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:50.392930 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1c8bb-029b-4fd9-889a-686e071e36ce None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:50.393217 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1c8bb-029b-4fd9-889a-686e071e36ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:50.411250 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f1c8bb-029b-4fd9-889a-686e071e36ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:31:50.412735 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f1c8bb-029b-4fd9-889a-686e071e36ce None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:31:50.428526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-32f1c8bb-029b-4fd9-889a-686e071e36ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:31:50.428526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:31:50.428526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:31:50.428526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:31:50.428526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:31:50.428526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:31:50.428526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:31:50.428526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:31:50.428526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:31:50.428526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:31:50.428526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:31:50.428526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:31:50.428526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:31:50.428526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:31:50.428526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:31:50.428526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:31:50.429394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:31:50.429394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:31:50.429394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:31:50.429394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:31:50.429394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:31:50.429394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:31:50.429394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:31:50.429394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:31:50.429394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:31:50.429394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:31:50.429394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:31:50.429394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:31:50.429394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:31:50.429394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:31:50.429394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:31:50.429394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:31:50.429394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:31:50.430264 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:31:50.430264 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:31:50.430264 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:31:50.430264 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 9/18] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:31:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:31:50.435315 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1c8bb-029b-4fd9-889a-686e071e36ce None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:31:50.450955 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb66a3-0c44-4fdd-ad99-e919010bfece None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:31:50.457214 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:31:50.485214 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5370f572-5a3c-43c0-a6fb-1e8f2addb32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:31:50.485853 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370f572-5a3c-43c0-a6fb-1e8f2addb32a None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:50.486135 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370f572-5a3c-43c0-a6fb-1e8f2addb32a None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:50.486411 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370f572-5a3c-43c0-a6fb-1e8f2addb32a None admin] PATH_INFO: `/v3/domains/default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:50.487531 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5370f572-5a3c-43c0-a6fb-1e8f2addb32a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:31:50.489014 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5370f572-5a3c-43c0-a6fb-1e8f2addb32a None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:31:50.490570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 10/19] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:31:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:31:50.496233 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5370f572-5a3c-43c0-a6fb-1e8f2addb32a None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:31:50.535064 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1c8bb-029b-4fd9-889a-686e071e36ce None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:31:50.570293 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befda8a2-a15f-4814-85fd-533d566c6a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:31:50.570825 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befda8a2-a15f-4814-85fd-533d566c6a03 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:50.571117 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befda8a2-a15f-4814-85fd-533d566c6a03 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:50.571395 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befda8a2-a15f-4814-85fd-533d566c6a03 None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:50.572065 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befda8a2-a15f-4814-85fd-533d566c6a03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:31:50.572372 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befda8a2-a15f-4814-85fd-533d566c6a03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:31:50.573748 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befda8a2-a15f-4814-85fd-533d566c6a03 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:31:50.575325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:31:50.591097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 10/20] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:31:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:31:50.594981 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5370f572-5a3c-43c0-a6fb-1e8f2addb32a None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:31:50.632324 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ec66d-e4fa-4f76-90bf-484635047924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:31:50.633010 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ec66d-e4fa-4f76-90bf-484635047924 None admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:50.633297 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ec66d-e4fa-4f76-90bf-484635047924 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:50.633581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ec66d-e4fa-4f76-90bf-484635047924 None admin] PATH_INFO: `/v3/domains/default/users/f09f98306cd54b068020428d74bd0957/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:50.635575 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ec66d-e4fa-4f76-90bf-484635047924 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=f09f98306cd54b068020428d74bd0957, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:31:50.637092 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ec66d-e4fa-4f76-90bf-484635047924 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:31:50.813872 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 11/21] 10.222.0.17 () {58 vars in 1361 bytes} [Mon Oct 17 03:31:50 2022] PUT /identity/v3/domains/default/users/f09f98306cd54b068020428d74bd0957/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:31:53.154303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6085a-00cf-4c2d-b454-ae8e95f6d8fc None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:53.154303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6085a-00cf-4c2d-b454-ae8e95f6d8fc None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:53.154303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6085a-00cf-4c2d-b454-ae8e95f6d8fc None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:53.154303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 11/22] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:31:53.161020 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0fc42-c7b4-4199-a29f-0143b2dfc27d None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:53.161887 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0fc42-c7b4-4199-a29f-0143b2dfc27d None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:53.162354 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0fc42-c7b4-4199-a29f-0143b2dfc27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:53.188970 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-3ce0fc42-c7b4-4199-a29f-0143b2dfc27d None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:31:53.190851 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-3ce0fc42-c7b4-4199-a29f-0143b2dfc27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:31:53.258355 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa94516-a7e2-44ce-b9ec-6385e7cd36b3 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:53.258985 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa94516-a7e2-44ce-b9ec-6385e7cd36b3 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:53.259448 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa94516-a7e2-44ce-b9ec-6385e7cd36b3 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:53.261001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 12/23] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:31:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:31:53.273172 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d383c9-5b36-45c2-959b-6c940c0da8c9 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:53.273763 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d383c9-5b36-45c2-959b-6c940c0da8c9 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:53.274238 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d383c9-5b36-45c2-959b-6c940c0da8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:53.283761 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 12/24] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:31:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 03:31:53.316782 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-c9d383c9-5b36-45c2-959b-6c940c0da8c9 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:31:53.318020 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c9d383c9-5b36-45c2-959b-6c940c0da8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:31:53.411350 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 13/25] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:31:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Oct 17 03:31:53.599492 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0fc42-c7b4-4199-a29f-0143b2dfc27d None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:31:53.688212 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:31:53.709616 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0fc42-c7b4-4199-a29f-0143b2dfc27d None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:31:53.743523 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807e91a-8621-4c64-810a-1bcad8ef287d None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:53.744213 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807e91a-8621-4c64-810a-1bcad8ef287d None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:53.744773 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807e91a-8621-4c64-810a-1bcad8ef287d None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:53.746376 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 14/26] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:31:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:31:53.749343 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbbf00c-65e1-4fd9-a490-b3b62fbb1213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:31:53.749991 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbf00c-65e1-4fd9-a490-b3b62fbb1213 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:53.750271 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbf00c-65e1-4fd9-a490-b3b62fbb1213 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:53.750542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbbf00c-65e1-4fd9-a490-b3b62fbb1213 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:53.758547 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ae0ce-6a52-4574-b633-9db16799e4b6 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:53.765622 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ae0ce-6a52-4574-b633-9db16799e4b6 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:53.765622 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ae0ce-6a52-4574-b633-9db16799e4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:53.765803 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbbf00c-65e1-4fd9-a490-b3b62fbb1213 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:31:53.770522 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-dfbbf00c-65e1-4fd9-a490-b3b62fbb1213 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:31:53.770522 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:31:53.770522 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:31:53.770522 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:31:53.770522 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:31:53.770522 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:31:53.770522 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:31:53.770522 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:31:53.770522 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:31:53.770522 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:31:53.770522 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:31:53.770522 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:31:53.770522 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:31:53.770522 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:31:53.770522 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:31:53.770522 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:31:53.771347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:31:53.771347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:31:53.771347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:31:53.771347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:31:53.771347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:31:53.771347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:31:53.771347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:31:53.771347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:31:53.771347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:31:53.771347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:31:53.771347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:31:53.771347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:31:53.771347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:31:53.771347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:31:53.771347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:31:53.771347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:31:53.772131 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:31:53.772131 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:31:53.772131 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:31:53.772131 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:31:53.772131 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:31:53.772131 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:31:53.772131 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:31:53.772131 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:31:53.772131 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:31:53.772131 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:31:53.772131 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:31:53.772131 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:31:53.772131 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 13/27] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Oct 17 03:31:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:31:53.780871 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbbf00c-65e1-4fd9-a490-b3b62fbb1213 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:31:53.789773 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-cc2ae0ce-6a52-4574-b633-9db16799e4b6 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:31:53.791225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-cc2ae0ce-6a52-4574-b633-9db16799e4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:31:53.800977 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:31:53.928038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbbf00c-65e1-4fd9-a490-b3b62fbb1213 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:31:53.965830 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f31a6c-c82d-4513-a4d6-abecbd957b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:31:53.966706 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f31a6c-c82d-4513-a4d6-abecbd957b17 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:53.966989 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f31a6c-c82d-4513-a4d6-abecbd957b17 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:53.967263 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f31a6c-c82d-4513-a4d6-abecbd957b17 None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:53.968490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f31a6c-c82d-4513-a4d6-abecbd957b17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:31:53.968795 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f31a6c-c82d-4513-a4d6-abecbd957b17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:31:53.969519 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 15/28] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:31:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 03:31:53.970385 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f31a6c-c82d-4513-a4d6-abecbd957b17 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:31:53.970873 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ae0ce-6a52-4574-b633-9db16799e4b6 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:31:53.985222 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 14/29] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:31:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:31:53.986742 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff044ce4-3b30-4e49-893e-bbb88b3d2e6c None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:53.987033 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff044ce4-3b30-4e49-893e-bbb88b3d2e6c None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:53.987309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff044ce4-3b30-4e49-893e-bbb88b3d2e6c None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:53.992974 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 15/30] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:31:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:31:53.996125 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:31:53.999068 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff044ce4-3b30-4e49-893e-bbb88b3d2e6c None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:31:54.160242 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ae0ce-6a52-4574-b633-9db16799e4b6 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:31:54.199149 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb594c1-8763-48f1-8d75-a68c85403824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:31:54.200179 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb594c1-8763-48f1-8d75-a68c85403824 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:54.200952 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb594c1-8763-48f1-8d75-a68c85403824 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:54.201439 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb594c1-8763-48f1-8d75-a68c85403824 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:54.211098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:31:54.213825 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb594c1-8763-48f1-8d75-a68c85403824 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:31:54.215602 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb594c1-8763-48f1-8d75-a68c85403824 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:31:54.232525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-3cb594c1-8763-48f1-8d75-a68c85403824 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:31:54.232525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:31:54.232525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:31:54.232525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:31:54.232525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:31:54.232525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:31:54.232525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:31:54.232525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:31:54.232525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:31:54.232525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:31:54.232525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:31:54.232525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:31:54.232525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:31:54.232525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:31:54.232525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:31:54.232525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:31:54.233402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:31:54.233402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:31:54.233402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:31:54.233402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:31:54.233402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:31:54.233402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:31:54.233402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:31:54.233402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:31:54.233402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:31:54.233402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:31:54.233402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:31:54.233402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:31:54.233402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:31:54.233402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:31:54.233402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:31:54.233402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:31:54.233402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:31:54.234205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:31:54.234205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:31:54.234205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:31:54.236439 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 16/31] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:31:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:31:54.237998 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ac45c-d031-4500-9d99-8371da81e700 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:54.238471 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ac45c-d031-4500-9d99-8371da81e700 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:54.239115 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ac45c-d031-4500-9d99-8371da81e700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:54.250365 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff044ce4-3b30-4e49-893e-bbb88b3d2e6c None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:31:54.260572 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-241ac45c-d031-4500-9d99-8371da81e700 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:31:54.262110 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-241ac45c-d031-4500-9d99-8371da81e700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:31:54.298294 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e850f-240d-448f-972d-7567ef216355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:31:54.298943 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e850f-240d-448f-972d-7567ef216355 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:54.299246 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e850f-240d-448f-972d-7567ef216355 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:54.299509 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e850f-240d-448f-972d-7567ef216355 None admin] PATH_INFO: `/v3/users/f09f98306cd54b068020428d74bd0957` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:54.300707 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793e850f-240d-448f-972d-7567ef216355 None admin] RBAC: Authorizing `identity:get_user(user_id=f09f98306cd54b068020428d74bd0957)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:31:54.302241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793e850f-240d-448f-972d-7567ef216355 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:31:54.304172 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 16/32] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:31:53 2022] GET /identity/v3/users/f09f98306cd54b068020428d74bd0957 => generated 247 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:31:54.308998 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e850f-240d-448f-972d-7567ef216355 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:31:54.355105 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 17/33] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:31:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:31:54.356209 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ac45c-d031-4500-9d99-8371da81e700 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:31:54.380805 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:31:54.429992 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e850f-240d-448f-972d-7567ef216355 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:31:54.476219 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2fdf9b-90b1-4cbf-8ea9-0a8eb12ec0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:31:54.477671 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fdf9b-90b1-4cbf-8ea9-0a8eb12ec0be None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:54.478100 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fdf9b-90b1-4cbf-8ea9-0a8eb12ec0be None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:54.478672 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fdf9b-90b1-4cbf-8ea9-0a8eb12ec0be None admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:54.479535 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2fdf9b-90b1-4cbf-8ea9-0a8eb12ec0be None admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:31:54.480282 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2fdf9b-90b1-4cbf-8ea9-0a8eb12ec0be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:31:54.481802 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2fdf9b-90b1-4cbf-8ea9-0a8eb12ec0be None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:31:54.503845 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 17/34] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:31:54.504956 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2fdf9b-90b1-4cbf-8ea9-0a8eb12ec0be None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:31:54.553552 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ac45c-d031-4500-9d99-8371da81e700 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:31:54.583106 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:31:54.594820 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b0a575-5f3d-4a8d-a0b9-759682f67a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:31:54.595450 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0a575-5f3d-4a8d-a0b9-759682f67a03 None admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:54.595730 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0a575-5f3d-4a8d-a0b9-759682f67a03 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:54.596034 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b0a575-5f3d-4a8d-a0b9-759682f67a03 None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:54.596722 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0a575-5f3d-4a8d-a0b9-759682f67a03 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:31:54.598126 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b0a575-5f3d-4a8d-a0b9-759682f67a03 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:31:54.724639 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2fdf9b-90b1-4cbf-8ea9-0a8eb12ec0be None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:31:54.746929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:31:54.761607 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075c4ad-7c42-4416-8da0-7c3f0ca0e5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:31:54.762177 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075c4ad-7c42-4416-8da0-7c3f0ca0e5b2 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:54.762459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075c4ad-7c42-4416-8da0-7c3f0ca0e5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:54.762800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075c4ad-7c42-4416-8da0-7c3f0ca0e5b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:54.763932 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075c4ad-7c42-4416-8da0-7c3f0ca0e5b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:31:54.765425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e075c4ad-7c42-4416-8da0-7c3f0ca0e5b2 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:31:54.771980 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 18/35] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:31:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:31:54.771980 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075c4ad-7c42-4416-8da0-7c3f0ca0e5b2 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:31:54.846802 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 18/36] 10.222.0.17 () {60 vars in 1131 bytes} [Mon Oct 17 03:31:54 2022] POST /identity/v3/roles => generated 225 bytes in 491 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:31:54.848389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2c9c8-b226-4de9-a338-5dc8b3e294e6 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:54.848679 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2c9c8-b226-4de9-a338-5dc8b3e294e6 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:54.848958 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef2c9c8-b226-4de9-a338-5dc8b3e294e6 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:54.850557 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 19/37] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:31:54.855664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef2c9c8-b226-4de9-a338-5dc8b3e294e6 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:31:54.891805 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e075c4ad-7c42-4416-8da0-7c3f0ca0e5b2 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:31:54.931372 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864829a1-972f-4550-a57c-6d3e51c0a2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:31:54.932157 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864829a1-972f-4550-a57c-6d3e51c0a2c0 None admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:54.932497 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864829a1-972f-4550-a57c-6d3e51c0a2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:54.932779 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864829a1-972f-4550-a57c-6d3e51c0a2c0 None admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:54.933454 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864829a1-972f-4550-a57c-6d3e51c0a2c0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:31:54.934915 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864829a1-972f-4550-a57c-6d3e51c0a2c0 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:31:54.978318 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef2c9c8-b226-4de9-a338-5dc8b3e294e6 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:31:55.015591 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f5ed6-dfb0-4417-a83c-8a6e4ed083ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:31:55.016153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f5ed6-dfb0-4417-a83c-8a6e4ed083ed None admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:55.016438 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f5ed6-dfb0-4417-a83c-8a6e4ed083ed None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:55.016716 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f5ed6-dfb0-4417-a83c-8a6e4ed083ed None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:55.017401 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f5ed6-dfb0-4417-a83c-8a6e4ed083ed None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:31:55.019003 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f5ed6-dfb0-4417-a83c-8a6e4ed083ed None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:31:55.052430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 19/38] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Oct 17 03:31:54 2022] POST /identity/v3/projects => generated 301 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:31:55.053800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864829a1-972f-4550-a57c-6d3e51c0a2c0 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:31:55.106446 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 20/39] 10.222.0.17 () {60 vars in 1131 bytes} [Mon Oct 17 03:31:54 2022] POST /identity/v3/roles => generated 219 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:31:55.107515 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ba957-6252-4408-a928-678f2074e0fd None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:55.107853 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ba957-6252-4408-a928-678f2074e0fd None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:55.108172 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ba957-6252-4408-a928-678f2074e0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:55.111588 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:31:55.134772 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-df2ba957-6252-4408-a928-678f2074e0fd None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:31:55.136535 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-df2ba957-6252-4408-a928-678f2074e0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:31:55.205315 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:31:55.215706 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864829a1-972f-4550-a57c-6d3e51c0a2c0 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:31:55.252964 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 21/40] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:31:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:31:55.254341 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1ac7b-c464-4d05-a04d-06d6be9d812b None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:55.254660 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1ac7b-c464-4d05-a04d-06d6be9d812b None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:55.254918 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1ac7b-c464-4d05-a04d-06d6be9d812b None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:55.256393 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 22/41] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:31:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:31:55.262562 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a7ebc-ce6e-4571-a049-a17d37837c5c None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:55.262891 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a7ebc-ce6e-4571-a049-a17d37837c5c None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:55.263115 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a7ebc-ce6e-4571-a049-a17d37837c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:55.266995 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aef957-a488-435c-b4b6-e793d1385bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:31:55.268337 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aef957-a488-435c-b4b6-e793d1385bc4 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:55.268785 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aef957-a488-435c-b4b6-e793d1385bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:55.269297 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aef957-a488-435c-b4b6-e793d1385bc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:55.270239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aef957-a488-435c-b4b6-e793d1385bc4 None admin] RBAC: Adding query filter params (role.id=611120afb3a544d0b5161da1b769a2f8, scope.domain.id=default, user.id=f09f98306cd54b068020428d74bd0957) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:31:55.270693 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aef957-a488-435c-b4b6-e793d1385bc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:31:55.274361 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aef957-a488-435c-b4b6-e793d1385bc4 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:31:55.285232 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 20/42] 10.222.0.17 () {56 vars in 1322 bytes} [Mon Oct 17 03:31:55 2022] GET /identity/v3/role_assignments?user.id=f09f98306cd54b068020428d74bd0957&scope.domain.id=default&role.id=611120afb3a544d0b5161da1b769a2f8 => generated 530 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:31:55.291705 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-550a7ebc-ce6e-4571-a049-a17d37837c5c None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:31:55.293423 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-550a7ebc-ce6e-4571-a049-a17d37837c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:31:55.377311 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 23/43] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:31:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 03:31:55.542045 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aef957-a488-435c-b4b6-e793d1385bc4 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:31:55.639052 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aef957-a488-435c-b4b6-e793d1385bc4 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:31:55.641882 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:31:55.663769 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a7ebc-ce6e-4571-a049-a17d37837c5c None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:31:55.672732 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc53f4-8b3c-422c-b361-cce335a464a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:31:55.673346 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc53f4-8b3c-422c-b361-cce335a464a6 None admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:55.673635 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc53f4-8b3c-422c-b361-cce335a464a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:55.673908 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc53f4-8b3c-422c-b361-cce335a464a6 None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:55.674592 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc53f4-8b3c-422c-b361-cce335a464a6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:31:55.676014 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc53f4-8b3c-422c-b361-cce335a464a6 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:31:55.763041 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a7ebc-ce6e-4571-a049-a17d37837c5c None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:31:55.765058 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:31:55.791514 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 21/44] 10.222.0.17 () {60 vars in 1131 bytes} [Mon Oct 17 03:31:55 2022] POST /identity/v3/roles => generated 223 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:31:55.796628 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e165ecea-3f17-4f5c-b6c4-400486a4dfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:31:55.797224 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165ecea-3f17-4f5c-b6c4-400486a4dfda None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:55.797509 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165ecea-3f17-4f5c-b6c4-400486a4dfda None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:55.797840 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e165ecea-3f17-4f5c-b6c4-400486a4dfda None admin] PATH_INFO: `/v3/domains/default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:55.799203 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e165ecea-3f17-4f5c-b6c4-400486a4dfda None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:31:55.800754 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e165ecea-3f17-4f5c-b6c4-400486a4dfda None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:31:55.803770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 24/45] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:31:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:31:55.808590 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc53f4-8b3c-422c-b361-cce335a464a6 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:31:55.917668 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc53f4-8b3c-422c-b361-cce335a464a6 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:31:55.953258 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59e734-8aed-4c97-a232-ab7d7620f273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:31:55.953885 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59e734-8aed-4c97-a232-ab7d7620f273 None admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:55.954171 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59e734-8aed-4c97-a232-ab7d7620f273 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:55.954451 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59e734-8aed-4c97-a232-ab7d7620f273 None admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:55.955128 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59e734-8aed-4c97-a232-ab7d7620f273 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:31:55.956606 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59e734-8aed-4c97-a232-ab7d7620f273 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:31:56.039024 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 22/46] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Oct 17 03:31:55 2022] POST /identity/v3/projects => generated 312 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:31:58.157074 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c081cc-9e1d-4006-8ef1-8668c7105617 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:58.157613 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c081cc-9e1d-4006-8ef1-8668c7105617 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:58.157613 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c081cc-9e1d-4006-8ef1-8668c7105617 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:58.158956 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 25/47] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:31:58.164213 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e31921-e85d-481a-b517-2b24839771de None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:58.164609 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e31921-e85d-481a-b517-2b24839771de None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:58.164777 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e31921-e85d-481a-b517-2b24839771de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:58.183091 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-52e31921-e85d-481a-b517-2b24839771de None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:31:58.184314 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-52e31921-e85d-481a-b517-2b24839771de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:31:58.194731 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:31:58.249164 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 23/48] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:31:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:31:58.512217 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c081cc-9e1d-4006-8ef1-8668c7105617 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:31:58.609654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c081cc-9e1d-4006-8ef1-8668c7105617 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:31:58.641699 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ae62d-6dd4-4c40-ac89-f2d86c74f60f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:31:58.642314 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ae62d-6dd4-4c40-ac89-f2d86c74f60f None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:58.642592 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ae62d-6dd4-4c40-ac89-f2d86c74f60f None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:58.642863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ae62d-6dd4-4c40-ac89-f2d86c74f60f None admin] PATH_INFO: `/v3/domains/default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:58.643984 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ae62d-6dd4-4c40-ac89-f2d86c74f60f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:31:58.645397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ae62d-6dd4-4c40-ac89-f2d86c74f60f None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:31:58.646911 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 26/49] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:31:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:31:58.652378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e31921-e85d-481a-b517-2b24839771de None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:31:58.749039 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e31921-e85d-481a-b517-2b24839771de None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:31:58.782495 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a3195b-ae0b-4c35-a0dc-4fbae66970ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:31:58.783050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3195b-ae0b-4c35-a0dc-4fbae66970ba None admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:31:58.783338 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3195b-ae0b-4c35-a0dc-4fbae66970ba None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:31:58.783614 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3195b-ae0b-4c35-a0dc-4fbae66970ba None admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:31:58.784295 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3195b-ae0b-4c35-a0dc-4fbae66970ba None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:31:58.785741 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a3195b-ae0b-4c35-a0dc-4fbae66970ba None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:31:58.895728 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:31:58.902093 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 24/50] 10.222.0.17 () {60 vars in 1140 bytes} [Mon Oct 17 03:31:58 2022] POST /identity/v3/projects => generated 298 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:32:01.034429 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76b324-35ca-4675-a013-580caa8df07c None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:01.034962 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76b324-35ca-4675-a013-580caa8df07c None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:01.034962 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76b324-35ca-4675-a013-580caa8df07c None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:01.036336 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 27/51] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:01.041402 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799ed43-01fb-4f4a-b4e8-4496dbb0d3c7 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:01.041774 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799ed43-01fb-4f4a-b4e8-4496dbb0d3c7 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:01.041973 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7799ed43-01fb-4f4a-b4e8-4496dbb0d3c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:01.060278 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-7799ed43-01fb-4f4a-b4e8-4496dbb0d3c7 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:01.062132 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-7799ed43-01fb-4f4a-b4e8-4496dbb0d3c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:01.126543 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 25/52] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:01.392637 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76b324-35ca-4675-a013-580caa8df07c None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:01.431975 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:01.490317 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76b324-35ca-4675-a013-580caa8df07c None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:01.522526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f3112-ddce-4047-85f6-48a307442536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:01.523148 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f3112-ddce-4047-85f6-48a307442536 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:01.523430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f3112-ddce-4047-85f6-48a307442536 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:01.523703 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f3112-ddce-4047-85f6-48a307442536 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:01.524839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54f3112-ddce-4047-85f6-48a307442536 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:01.526273 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54f3112-ddce-4047-85f6-48a307442536 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:01.527830 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 28/53] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:32:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:01.533291 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7799ed43-01fb-4f4a-b4e8-4496dbb0d3c7 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:01.631577 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7799ed43-01fb-4f4a-b4e8-4496dbb0d3c7 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:01.656987 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:01.664316 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9fa87c-9720-48e0-8c2e-ce4c724bc50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:01.664927 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9fa87c-9720-48e0-8c2e-ce4c724bc50d None admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:01.665214 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9fa87c-9720-48e0-8c2e-ce4c724bc50d None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:01.665495 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9fa87c-9720-48e0-8c2e-ce4c724bc50d None admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:01.666161 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9fa87c-9720-48e0-8c2e-ce4c724bc50d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:01.667612 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9fa87c-9720-48e0-8c2e-ce4c724bc50d None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:01.767949 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 26/54] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Oct 17 03:32:01 2022] POST /identity/v3/users => generated 275 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:32:05.055161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77c5b7-82fd-4590-8c0b-33a1e54b33e9 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:05.055161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77c5b7-82fd-4590-8c0b-33a1e54b33e9 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:05.055161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77c5b7-82fd-4590-8c0b-33a1e54b33e9 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:05.055161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 29/55] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:05.060579 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3765b-054a-46ba-ac06-82c2b1788796 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:05.060962 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3765b-054a-46ba-ac06-82c2b1788796 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:05.061143 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c3765b-054a-46ba-ac06-82c2b1788796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:05.068226 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3aaa3-f196-4b3e-9a02-aa19a0ed2e26 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:05.068511 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3aaa3-f196-4b3e-9a02-aa19a0ed2e26 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:05.068788 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c3aaa3-f196-4b3e-9a02-aa19a0ed2e26 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:05.070270 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 30/56] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:32:05.076053 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804e51a-8201-4ba5-b66f-ec02caf4dbc1 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:05.076381 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804e51a-8201-4ba5-b66f-ec02caf4dbc1 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:05.076659 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804e51a-8201-4ba5-b66f-ec02caf4dbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:05.086862 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-27c3765b-054a-46ba-ac06-82c2b1788796 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:05.089920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-27c3765b-054a-46ba-ac06-82c2b1788796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:05.103313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-0804e51a-8201-4ba5-b66f-ec02caf4dbc1 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:05.104781 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-0804e51a-8201-4ba5-b66f-ec02caf4dbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:05.175652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 31/57] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 03:32:05.205769 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 27/58] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:05.345304 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ff728-bc47-4720-b656-0dbd39123f6c None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:05.345895 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ff728-bc47-4720-b656-0dbd39123f6c None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:05.346365 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ff728-bc47-4720-b656-0dbd39123f6c None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:05.347943 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 32/59] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:05.353257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6efee-4243-4f40-9fbf-5b5f05d93a68 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:05.353799 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6efee-4243-4f40-9fbf-5b5f05d93a68 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:05.354264 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6efee-4243-4f40-9fbf-5b5f05d93a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:05.384718 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-abc6efee-4243-4f40-9fbf-5b5f05d93a68 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:05.386291 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-abc6efee-4243-4f40-9fbf-5b5f05d93a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:05.461835 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 28/60] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:05.511329 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ff728-bc47-4720-b656-0dbd39123f6c None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:05.521197 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6efee-4243-4f40-9fbf-5b5f05d93a68 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:05.567644 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:05.573384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:05.654145 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6efee-4243-4f40-9fbf-5b5f05d93a68 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:05.671489 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ff728-bc47-4720-b656-0dbd39123f6c None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:05.697914 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b37dc98-60cc-493f-9d3d-ecceac94b528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:05.698980 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37dc98-60cc-493f-9d3d-ecceac94b528 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:05.699477 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37dc98-60cc-493f-9d3d-ecceac94b528 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:05.700050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37dc98-60cc-493f-9d3d-ecceac94b528 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:05.710140 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7310d9-0579-4d8d-a35a-ac1c88804b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:05.710986 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7310d9-0579-4d8d-a35a-ac1c88804b8b None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:05.711415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7310d9-0579-4d8d-a35a-ac1c88804b8b None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:05.711964 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7310d9-0579-4d8d-a35a-ac1c88804b8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:05.714523 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b37dc98-60cc-493f-9d3d-ecceac94b528 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:05.720651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-6b37dc98-60cc-493f-9d3d-ecceac94b528 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 03:32:05.720651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:05.720651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:05.720651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:05.720651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:05.720651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:05.720651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:05.720651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:05.720651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:05.720651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:05.720651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:05.720651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:05.720651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:05.720651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:05.720651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:05.720651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:05.721740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:05.721740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:05.721740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:05.721740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:05.721740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:05.721740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:05.721740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:05.721740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:05.721740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:05.721740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:05.721740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:05.721740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:05.721740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:05.721740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:05.721740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:05.721740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:05.722681 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:05.722681 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:05.722681 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:05.722681 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:05.722681 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:05.722681 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:05.722681 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:05.722681 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:05.722681 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:05.722681 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:05.722681 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 03:32:05.722681 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:05.724901 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 29/61] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:32:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:05.728657 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7310d9-0579-4d8d-a35a-ac1c88804b8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:05.730967 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f6052-d422-4d91-b441-b7d6ef76167c None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:05.731254 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f6052-d422-4d91-b441-b7d6ef76167c None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:05.731534 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f6052-d422-4d91-b441-b7d6ef76167c None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:05.733430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 30/62] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:05.738493 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f6052-d422-4d91-b441-b7d6ef76167c None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:05.741130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-4f7310d9-0579-4d8d-a35a-ac1c88804b8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:32:05.741130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:05.741130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:05.741130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:05.741130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:05.741130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:05.741130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:05.741130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:05.741130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:05.741130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:05.741130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:05.741130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:05.741130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:05.741130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:05.741130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:05.741130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:05.741931 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:05.741931 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:05.741931 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:05.741931 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:05.741931 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:05.741931 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:05.741931 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:05.741931 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:05.741931 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:05.741931 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:05.741931 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:05.741931 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:05.741931 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:05.741931 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:05.741931 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:05.741931 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:05.742674 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:05.742674 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:05.742674 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:05.742674 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:05.742674 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:05.742674 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:05.742674 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:05.742674 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:05.742674 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:05.742674 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:05.742674 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:32:05.742674 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:05.745177 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 33/63] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Oct 17 03:32:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:05.751285 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365127ba-cc2b-4763-9ac9-a5454ce75891 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:05.751794 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365127ba-cc2b-4763-9ac9-a5454ce75891 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:05.752253 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365127ba-cc2b-4763-9ac9-a5454ce75891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:05.771003 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-365127ba-cc2b-4763-9ac9-a5454ce75891 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:05.772611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-365127ba-cc2b-4763-9ac9-a5454ce75891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:05.843403 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f6052-d422-4d91-b441-b7d6ef76167c None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:05.845334 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 34/64] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:05.845334 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365127ba-cc2b-4763-9ac9-a5454ce75891 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:05.852951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:05.878447 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:05.879493 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bcc0be-e675-4b39-a52d-3a2e63243f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:05.880161 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bcc0be-e675-4b39-a52d-3a2e63243f3e None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:05.880446 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bcc0be-e675-4b39-a52d-3a2e63243f3e None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:05.880884 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bcc0be-e675-4b39-a52d-3a2e63243f3e None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:05.881554 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bcc0be-e675-4b39-a52d-3a2e63243f3e None admin] RBAC: Adding query filter params (name=member) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:05.881861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bcc0be-e675-4b39-a52d-3a2e63243f3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:05.883231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bcc0be-e675-4b39-a52d-3a2e63243f3e None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:05.895109 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 31/65] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Oct 17 03:32:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:05.896229 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e642c-eb0c-4a07-a520-16c052bf0865 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:05.896516 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e642c-eb0c-4a07-a520-16c052bf0865 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:05.896796 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e642c-eb0c-4a07-a520-16c052bf0865 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:05.898222 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 32/66] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:05.903259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e642c-eb0c-4a07-a520-16c052bf0865 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:06.019233 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e642c-eb0c-4a07-a520-16c052bf0865 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:06.055153 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675cdc3-097b-44e1-83f6-8edef86c84a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:06.055819 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675cdc3-097b-44e1-83f6-8edef86c84a5 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:06.056098 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675cdc3-097b-44e1-83f6-8edef86c84a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:06.056381 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675cdc3-097b-44e1-83f6-8edef86c84a5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:06.062937 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365127ba-cc2b-4763-9ac9-a5454ce75891 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:06.067123 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a675cdc3-097b-44e1-83f6-8edef86c84a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:06.072113 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-a675cdc3-097b-44e1-83f6-8edef86c84a5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 03:32:06.072113 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:06.072113 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:06.072113 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:06.072113 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:06.072113 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:06.072113 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:06.072113 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:06.072113 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:06.072113 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:06.072113 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:06.072113 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:06.072113 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:06.072113 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:06.072113 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:06.072113 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:06.073146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:06.073146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:06.073146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:06.073146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:06.073146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:06.073146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:06.073146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:06.073146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:06.073146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:06.073146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:06.073146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:06.073146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:06.073146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:06.073146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:06.073146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:06.073146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:06.077269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:06.077269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:06.077269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:06.077269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:06.077269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:06.077269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:06.077269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:06.077269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:06.077269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:06.077269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:06.077269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 03:32:06.077269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:06.077269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 33/67] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Oct 17 03:32:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:06.077269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675cdc3-097b-44e1-83f6-8edef86c84a5 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:06.102216 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f111581-4214-4f8b-ab84-0354966ee312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:06.102833 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f111581-4214-4f8b-ab84-0354966ee312 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:06.103118 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f111581-4214-4f8b-ab84-0354966ee312 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:06.103398 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f111581-4214-4f8b-ab84-0354966ee312 None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:06.104103 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f111581-4214-4f8b-ab84-0354966ee312 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:06.104410 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f111581-4214-4f8b-ab84-0354966ee312 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:06.105780 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f111581-4214-4f8b-ab84-0354966ee312 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:06.105904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:06.117791 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 35/68] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:32:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:06.119033 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e1a99-1d4c-4d17-81ba-58bc01838469 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:06.119322 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e1a99-1d4c-4d17-81ba-58bc01838469 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:06.119602 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86e1a99-1d4c-4d17-81ba-58bc01838469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:06.136944 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-d86e1a99-1d4c-4d17-81ba-58bc01838469 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:06.138142 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-d86e1a99-1d4c-4d17-81ba-58bc01838469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:06.178093 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675cdc3-097b-44e1-83f6-8edef86c84a5 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:06.204855 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 36/69] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:06.205685 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86e1a99-1d4c-4d17-81ba-58bc01838469 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:06.225765 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:06.225918 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d65fb-aeb3-4e38-a396-98b53ab35238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:06.226455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d65fb-aeb3-4e38-a396-98b53ab35238 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:06.226738 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d65fb-aeb3-4e38-a396-98b53ab35238 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:06.227016 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17d65fb-aeb3-4e38-a396-98b53ab35238 None admin] PATH_INFO: `/v3/users/6704615fe74b48a4a33d01b3505139b8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:06.241551 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17d65fb-aeb3-4e38-a396-98b53ab35238 None admin] RBAC: Authorizing `identity:get_user(user_id=6704615fe74b48a4a33d01b3505139b8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:06.243028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17d65fb-aeb3-4e38-a396-98b53ab35238 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:06.244698 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 34/70] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:32:06 2022] GET /identity/v3/users/6704615fe74b48a4a33d01b3505139b8 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:06.245473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d65fb-aeb3-4e38-a396-98b53ab35238 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:06.309131 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86e1a99-1d4c-4d17-81ba-58bc01838469 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:06.340813 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:06.346145 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc8983-ec35-478e-bb88-7df56375c36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:06.346759 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc8983-ec35-478e-bb88-7df56375c36b None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:06.347046 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc8983-ec35-478e-bb88-7df56375c36b None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:06.347317 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc8983-ec35-478e-bb88-7df56375c36b None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:06.348068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc8983-ec35-478e-bb88-7df56375c36b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:06.353690 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc8983-ec35-478e-bb88-7df56375c36b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:06.353690 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc8983-ec35-478e-bb88-7df56375c36b None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:06.357809 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 37/71] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:32:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:06.358528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc8983-ec35-478e-bb88-7df56375c36b None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:06.360088 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17d65fb-aeb3-4e38-a396-98b53ab35238 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:06.398516 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ec846-9163-4ed6-bbdf-560fca11f95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:06.399434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ec846-9163-4ed6-bbdf-560fca11f95e None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:06.399919 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ec846-9163-4ed6-bbdf-560fca11f95e None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:06.400366 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ec846-9163-4ed6-bbdf-560fca11f95e None admin] PATH_INFO: `/v3/roles/member` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:06.411490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601ec846-9163-4ed6-bbdf-560fca11f95e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:06.416624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-601ec846-9163-4ed6-bbdf-560fca11f95e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 03:32:06.416624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:06.416624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:06.416624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:06.416624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:06.416624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:06.416624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:06.416624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:06.416624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:06.416624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:06.416624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:06.416624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:06.416624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:06.416624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:06.416624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:06.416624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:06.417448 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:06.417448 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:06.417448 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:06.417448 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:06.417448 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:06.417448 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:06.417448 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:06.417448 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:06.417448 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:06.417448 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:06.417448 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:06.417448 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:06.417448 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:06.417448 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:06.417448 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:06.417448 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:06.418187 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:06.418187 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:06.418187 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:06.418187 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:06.418187 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:06.418187 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:06.418187 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:06.418187 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:06.418187 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:06.418187 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:06.418187 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 03:32:06.418187 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:06.418187 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 35/72] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:32:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:06.419131 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ec846-9163-4ed6-bbdf-560fca11f95e None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:06.450398 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:06.470062 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc8983-ec35-478e-bb88-7df56375c36b None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:06.503619 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b2a8a-214c-48bd-8f42-614ea07862af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:06.504289 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b2a8a-214c-48bd-8f42-614ea07862af None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:06.504567 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b2a8a-214c-48bd-8f42-614ea07862af None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:06.504842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b2a8a-214c-48bd-8f42-614ea07862af None admin] PATH_INFO: `/v3/users/f09f98306cd54b068020428d74bd0957` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:06.506210 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b2a8a-214c-48bd-8f42-614ea07862af None admin] RBAC: Authorizing `identity:get_user(user_id=f09f98306cd54b068020428d74bd0957)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:06.507643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b2a8a-214c-48bd-8f42-614ea07862af None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:06.509265 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 38/73] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:32:06 2022] GET /identity/v3/users/f09f98306cd54b068020428d74bd0957 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:06.510028 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b2a8a-214c-48bd-8f42-614ea07862af None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:06.529035 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ec846-9163-4ed6-bbdf-560fca11f95e None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:06.562844 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b782d-4073-43ad-9f9c-e343687a9da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:06.563403 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b782d-4073-43ad-9f9c-e343687a9da5 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:06.563684 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b782d-4073-43ad-9f9c-e343687a9da5 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:06.563989 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9b782d-4073-43ad-9f9c-e343687a9da5 None admin] PATH_INFO: `/v3/projects/bc8197db3d20408990645225920f63d7` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:06.565114 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9b782d-4073-43ad-9f9c-e343687a9da5 None admin] RBAC: Authorizing `identity:get_project(project_id=bc8197db3d20408990645225920f63d7)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:06.566532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9b782d-4073-43ad-9f9c-e343687a9da5 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:06.568325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 36/74] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:32:06 2022] GET /identity/v3/projects/bc8197db3d20408990645225920f63d7 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:06.572768 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b782d-4073-43ad-9f9c-e343687a9da5 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:06.587851 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:06.611164 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b2a8a-214c-48bd-8f42-614ea07862af None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:06.645544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb0480-555e-4f05-81ee-09b02dca2f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:06.646175 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb0480-555e-4f05-81ee-09b02dca2f11 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:06.646463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb0480-555e-4f05-81ee-09b02dca2f11 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:06.646743 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb0480-555e-4f05-81ee-09b02dca2f11 None admin] PATH_INFO: `/v3/users/6704615fe74b48a4a33d01b3505139b8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:06.647922 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb0480-555e-4f05-81ee-09b02dca2f11 None admin] RBAC: Authorizing `identity:get_user(user_id=6704615fe74b48a4a33d01b3505139b8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:06.649349 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb0480-555e-4f05-81ee-09b02dca2f11 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:06.650938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 39/75] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:32:06 2022] GET /identity/v3/users/6704615fe74b48a4a33d01b3505139b8 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:06.651697 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb0480-555e-4f05-81ee-09b02dca2f11 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:06.668335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:06.670733 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9b782d-4073-43ad-9f9c-e343687a9da5 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:06.705767 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c21aa-ba94-4553-9f86-c71b8ab14902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:06.706381 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c21aa-ba94-4553-9f86-c71b8ab14902 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:06.706662 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c21aa-ba94-4553-9f86-c71b8ab14902 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:06.706933 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c21aa-ba94-4553-9f86-c71b8ab14902 None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:06.707598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c21aa-ba94-4553-9f86-c71b8ab14902 None admin] RBAC: Adding query filter params (name=member) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:06.707928 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c21aa-ba94-4553-9f86-c71b8ab14902 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:06.709335 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c21aa-ba94-4553-9f86-c71b8ab14902 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:06.718062 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 37/76] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Oct 17 03:32:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:06.718830 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c21aa-ba94-4553-9f86-c71b8ab14902 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:06.756360 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb0480-555e-4f05-81ee-09b02dca2f11 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:06.791544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a5ba1-85a8-4b75-bef0-287e8dd29576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:06.792225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a5ba1-85a8-4b75-bef0-287e8dd29576 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:06.792507 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a5ba1-85a8-4b75-bef0-287e8dd29576 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:06.792780 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a5ba1-85a8-4b75-bef0-287e8dd29576 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:06.793468 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:06.793900 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a5ba1-85a8-4b75-bef0-287e8dd29576 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:06.795360 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61a5ba1-85a8-4b75-bef0-287e8dd29576 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:06.797004 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 40/77] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:32:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:06.800013 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a5ba1-85a8-4b75-bef0-287e8dd29576 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:06.822516 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c21aa-ba94-4553-9f86-c71b8ab14902 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:06.857969 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4813190-99c8-4826-8c91-86b1326a9863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:06.858609 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4813190-99c8-4826-8c91-86b1326a9863 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:06.858897 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4813190-99c8-4826-8c91-86b1326a9863 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:06.859226 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4813190-99c8-4826-8c91-86b1326a9863 None admin] PATH_INFO: `/v3/projects/bc8197db3d20408990645225920f63d7` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:06.860417 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4813190-99c8-4826-8c91-86b1326a9863 None admin] RBAC: Authorizing `identity:get_project(project_id=bc8197db3d20408990645225920f63d7)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:06.861860 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4813190-99c8-4826-8c91-86b1326a9863 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:06.863500 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 38/78] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:32:06 2022] GET /identity/v3/projects/bc8197db3d20408990645225920f63d7 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:06.864366 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4813190-99c8-4826-8c91-86b1326a9863 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:06.887307 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:06.903671 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a5ba1-85a8-4b75-bef0-287e8dd29576 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:06.938412 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85846ce6-fac8-4723-8d6f-a21e200b5548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:06.939084 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85846ce6-fac8-4723-8d6f-a21e200b5548 None admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:06.939368 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85846ce6-fac8-4723-8d6f-a21e200b5548 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:06.939644 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85846ce6-fac8-4723-8d6f-a21e200b5548 None admin] PATH_INFO: `/v3/projects/bc8197db3d20408990645225920f63d7/users/6704615fe74b48a4a33d01b3505139b8/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:06.941641 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85846ce6-fac8-4723-8d6f-a21e200b5548 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc8197db3d20408990645225920f63d7, user_id=6704615fe74b48a4a33d01b3505139b8, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:06.943069 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85846ce6-fac8-4723-8d6f-a21e200b5548 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:06.962680 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4813190-99c8-4826-8c91-86b1326a9863 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:06.995631 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ee3b5-f0a3-42f5-accb-2dc68f06b884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:06.996287 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ee3b5-f0a3-42f5-accb-2dc68f06b884 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:06.996570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ee3b5-f0a3-42f5-accb-2dc68f06b884 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:06.996881 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ee3b5-f0a3-42f5-accb-2dc68f06b884 None admin] PATH_INFO: `/v3/projects/bc8197db3d20408990645225920f63d7` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:06.998013 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ee3b5-f0a3-42f5-accb-2dc68f06b884 None admin] RBAC: Authorizing `identity:get_project(project_id=bc8197db3d20408990645225920f63d7)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:06.999437 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ee3b5-f0a3-42f5-accb-2dc68f06b884 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:07.001129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 39/79] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:32:06 2022] GET /identity/v3/projects/bc8197db3d20408990645225920f63d7 => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:07.001965 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ee3b5-f0a3-42f5-accb-2dc68f06b884 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:07.014574 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:07.020614 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 41/80] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Oct 17 03:32:06 2022] PUT /identity/v3/projects/bc8197db3d20408990645225920f63d7/users/6704615fe74b48a4a33d01b3505139b8/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:32:07.020614 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85846ce6-fac8-4723-8d6f-a21e200b5548 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:07.112559 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ee3b5-f0a3-42f5-accb-2dc68f06b884 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:07.129407 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85846ce6-fac8-4723-8d6f-a21e200b5548 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:07.155914 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925750b-e233-46cd-b12c-13694a87e423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:07.156580 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925750b-e233-46cd-b12c-13694a87e423 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:07.156868 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925750b-e233-46cd-b12c-13694a87e423 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:07.157152 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925750b-e233-46cd-b12c-13694a87e423 None admin] PATH_INFO: `/v3/users/6704615fe74b48a4a33d01b3505139b8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:07.158374 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e925750b-e233-46cd-b12c-13694a87e423 None admin] RBAC: Authorizing `identity:get_user(user_id=6704615fe74b48a4a33d01b3505139b8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:07.159883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e925750b-e233-46cd-b12c-13694a87e423 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:07.161609 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 40/81] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:32:07 2022] GET /identity/v3/users/6704615fe74b48a4a33d01b3505139b8 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:07.161609 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925750b-e233-46cd-b12c-13694a87e423 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:07.174142 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612130b-9c2e-4dac-a9b9-f30bb214dfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:07.175065 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612130b-9c2e-4dac-a9b9-f30bb214dfcd None admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:07.175523 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612130b-9c2e-4dac-a9b9-f30bb214dfcd None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:07.176036 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612130b-9c2e-4dac-a9b9-f30bb214dfcd None admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:07.176880 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612130b-9c2e-4dac-a9b9-f30bb214dfcd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:07.178591 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612130b-9c2e-4dac-a9b9-f30bb214dfcd None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:07.246528 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:07.256329 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:07.261518 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 42/82] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Oct 17 03:32:07 2022] POST /identity/v3/users => generated 289 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:32:07.262719 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612130b-9c2e-4dac-a9b9-f30bb214dfcd None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:07.270691 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925750b-e233-46cd-b12c-13694a87e423 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:07.311601 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a83e4-6259-4327-9b27-2cf254448938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:07.312449 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a83e4-6259-4327-9b27-2cf254448938 None admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:07.313121 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a83e4-6259-4327-9b27-2cf254448938 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:07.313418 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a83e4-6259-4327-9b27-2cf254448938 None admin] PATH_INFO: `/v3/projects/bc8197db3d20408990645225920f63d7/users/f09f98306cd54b068020428d74bd0957/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:07.316972 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a83e4-6259-4327-9b27-2cf254448938 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc8197db3d20408990645225920f63d7, user_id=f09f98306cd54b068020428d74bd0957, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:07.317747 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a83e4-6259-4327-9b27-2cf254448938 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:07.389870 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612130b-9c2e-4dac-a9b9-f30bb214dfcd None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:07.426261 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc5dcc-bb0f-4b6e-b7e6-8f2886f3f8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:07.426649 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc5dcc-bb0f-4b6e-b7e6-8f2886f3f8d3 None admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:07.426817 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc5dcc-bb0f-4b6e-b7e6-8f2886f3f8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:07.428127 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc5dcc-bb0f-4b6e-b7e6-8f2886f3f8d3 None admin] PATH_INFO: `/v3/projects/bc8197db3d20408990645225920f63d7/users/6704615fe74b48a4a33d01b3505139b8/roles/609a4bf154ff4dcabb408b3789d28983` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:07.430619 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc5dcc-bb0f-4b6e-b7e6-8f2886f3f8d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc8197db3d20408990645225920f63d7, user_id=6704615fe74b48a4a33d01b3505139b8, role_id=609a4bf154ff4dcabb408b3789d28983)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:07.432426 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc5dcc-bb0f-4b6e-b7e6-8f2886f3f8d3 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:07.434581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 41/83] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Oct 17 03:32:07 2022] PUT /identity/v3/projects/bc8197db3d20408990645225920f63d7/users/f09f98306cd54b068020428d74bd0957/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:32:07.436247 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a83e4-6259-4327-9b27-2cf254448938 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:07.532030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 43/84] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Oct 17 03:32:07 2022] PUT /identity/v3/projects/bc8197db3d20408990645225920f63d7/users/6704615fe74b48a4a33d01b3505139b8/roles/609a4bf154ff4dcabb408b3789d28983 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:32:07.571989 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a83e4-6259-4327-9b27-2cf254448938 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:07.618997 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4748d43-32b4-4dbe-afa6-e7c62efde455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:07.619679 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4748d43-32b4-4dbe-afa6-e7c62efde455 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:07.619997 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4748d43-32b4-4dbe-afa6-e7c62efde455 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:07.620277 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4748d43-32b4-4dbe-afa6-e7c62efde455 None admin] PATH_INFO: `/v3/projects/b423bebfa6774da6a57f373ee36fbebb` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:07.621581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4748d43-32b4-4dbe-afa6-e7c62efde455 None admin] RBAC: Authorizing `identity:get_project(project_id=b423bebfa6774da6a57f373ee36fbebb)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:07.623671 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4748d43-32b4-4dbe-afa6-e7c62efde455 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:07.625847 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 42/85] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:32:07 2022] GET /identity/v3/projects/b423bebfa6774da6a57f373ee36fbebb => generated 312 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:07.632347 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc5dcc-bb0f-4b6e-b7e6-8f2886f3f8d3 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:07.710882 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:07.759726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc5dcc-bb0f-4b6e-b7e6-8f2886f3f8d3 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:07.799812 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af6a8d-a4a8-4dac-94fb-5d16694ce0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:07.800570 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af6a8d-a4a8-4dac-94fb-5d16694ce0c4 None admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:07.800919 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af6a8d-a4a8-4dac-94fb-5d16694ce0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:07.801258 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af6a8d-a4a8-4dac-94fb-5d16694ce0c4 None admin] PATH_INFO: `/v3/projects/b423bebfa6774da6a57f373ee36fbebb/users/6704615fe74b48a4a33d01b3505139b8/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:07.803414 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af6a8d-a4a8-4dac-94fb-5d16694ce0c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=b423bebfa6774da6a57f373ee36fbebb, user_id=6704615fe74b48a4a33d01b3505139b8, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:07.805016 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af6a8d-a4a8-4dac-94fb-5d16694ce0c4 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:07.894428 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 44/86] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Oct 17 03:32:07 2022] PUT /identity/v3/projects/b423bebfa6774da6a57f373ee36fbebb/users/6704615fe74b48a4a33d01b3505139b8/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:32:09.945573 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b2b75-1eca-4ecf-9abb-5d88971f786c None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:09.946571 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b2b75-1eca-4ecf-9abb-5d88971f786c None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:09.947070 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73b2b75-1eca-4ecf-9abb-5d88971f786c None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:09.948918 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 43/87] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:09.954267 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9b90e-d96a-45fe-ae51-1a7df42814f0 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:09.955039 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9b90e-d96a-45fe-ae51-1a7df42814f0 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:09.955522 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9b90e-d96a-45fe-ae51-1a7df42814f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:09.980716 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-e6c9b90e-d96a-45fe-ae51-1a7df42814f0 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:09.982269 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-e6c9b90e-d96a-45fe-ae51-1a7df42814f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:10.058337 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 45/88] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:10.367277 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967607b-03de-4ce4-a339-640683e81b13 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:10.367277 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967607b-03de-4ce4-a339-640683e81b13 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:10.367277 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967607b-03de-4ce4-a339-640683e81b13 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:10.368078 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 44/89] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:10.377619 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863160f-c147-46e6-a07f-bcb9c1f5ad4b None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:10.378048 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298bde7-8a20-4dec-a2da-343d6a897a1b None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:10.378344 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298bde7-8a20-4dec-a2da-343d6a897a1b None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:10.378594 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863160f-c147-46e6-a07f-bcb9c1f5ad4b None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:10.378692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298bde7-8a20-4dec-a2da-343d6a897a1b None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:10.379148 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863160f-c147-46e6-a07f-bcb9c1f5ad4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:10.380022 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 46/90] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:10.386620 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01027bf-d38a-4982-8378-1dd3b619274e None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:10.386906 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01027bf-d38a-4982-8378-1dd3b619274e None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:10.387179 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01027bf-d38a-4982-8378-1dd3b619274e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:10.399939 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:10.412470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-b863160f-c147-46e6-a07f-bcb9c1f5ad4b None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:10.414118 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-b863160f-c147-46e6-a07f-bcb9c1f5ad4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:10.424859 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-c01027bf-d38a-4982-8378-1dd3b619274e None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:10.426041 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c01027bf-d38a-4982-8378-1dd3b619274e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:10.534411 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:10.548165 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 45/91] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:10.565979 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 47/92] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:10.598911 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b863160f-c147-46e6-a07f-bcb9c1f5ad4b None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:10.770607 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b863160f-c147-46e6-a07f-bcb9c1f5ad4b None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:10.827331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb5840b-ac1a-4fc7-ae07-12d1aa204013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:10.828375 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5840b-ac1a-4fc7-ae07-12d1aa204013 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:10.828874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5840b-ac1a-4fc7-ae07-12d1aa204013 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:10.829337 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb5840b-ac1a-4fc7-ae07-12d1aa204013 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:10.838349 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:10.851901 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb5840b-ac1a-4fc7-ae07-12d1aa204013 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:10.857755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-efb5840b-ac1a-4fc7-ae07-12d1aa204013 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 03:32:10.857755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:10.857755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:10.857755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:10.857755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:10.857755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:10.857755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:10.857755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:10.857755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:10.857755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:10.857755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:10.857755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:10.857755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:10.857755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:10.857755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:10.857755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:10.864815 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:10.864815 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:10.864815 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:10.864815 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:10.864815 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:10.864815 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:10.864815 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:10.864815 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:10.864815 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:10.864815 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:10.864815 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:10.864815 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:10.864815 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:10.864815 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:10.864815 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:10.864815 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:10.865551 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:10.865551 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:10.865551 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:10.865551 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:10.865551 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:10.865551 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:10.865551 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:10.865551 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:10.865551 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:10.865551 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:10.865551 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 03:32:10.865551 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:10.865551 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 46/93] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:32:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:10.866167 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01027bf-d38a-4982-8378-1dd3b619274e None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:10.910534 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16f863-ab2d-4f27-8702-f041ee584df8 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:10.911306 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16f863-ab2d-4f27-8702-f041ee584df8 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:10.915833 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b16f863-ab2d-4f27-8702-f041ee584df8 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:10.917832 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 47/94] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:10 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:32:10.929851 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d163b06-fafa-43b5-9da0-affa987fc83f None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:10.930421 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d163b06-fafa-43b5-9da0-affa987fc83f None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:10.930888 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d163b06-fafa-43b5-9da0-affa987fc83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:10.970763 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-7d163b06-fafa-43b5-9da0-affa987fc83f None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:10.972745 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-7d163b06-fafa-43b5-9da0-affa987fc83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:11.089407 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 48/95] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:11.091750 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d163b06-fafa-43b5-9da0-affa987fc83f None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:11.107128 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01027bf-d38a-4982-8378-1dd3b619274e None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:11.188168 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6801b1-7a0b-4f8d-a560-01e32c60e58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:11.189180 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6801b1-7a0b-4f8d-a560-01e32c60e58d None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:11.189651 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6801b1-7a0b-4f8d-a560-01e32c60e58d None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:11.190112 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6801b1-7a0b-4f8d-a560-01e32c60e58d None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:11.190971 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6801b1-7a0b-4f8d-a560-01e32c60e58d None admin] RBAC: Adding query filter params (name=member) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:11.191465 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6801b1-7a0b-4f8d-a560-01e32c60e58d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:11.194351 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6801b1-7a0b-4f8d-a560-01e32c60e58d None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:11.214574 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 48/96] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Oct 17 03:32:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:11.219313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6801b1-7a0b-4f8d-a560-01e32c60e58d None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:11.276506 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d163b06-fafa-43b5-9da0-affa987fc83f None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:11.304215 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:11.311916 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:11.330049 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26980f0d-895d-420f-a3f4-8ea4f4fedefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:11.331031 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26980f0d-895d-420f-a3f4-8ea4f4fedefe None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:11.331506 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26980f0d-895d-420f-a3f4-8ea4f4fedefe None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:11.332017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26980f0d-895d-420f-a3f4-8ea4f4fedefe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:11.357780 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26980f0d-895d-420f-a3f4-8ea4f4fedefe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:11.363062 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-26980f0d-895d-420f-a3f4-8ea4f4fedefe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 03:32:11.363062 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:11.363062 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:11.363062 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:11.363062 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:11.363062 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:11.363062 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:11.363062 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:11.363062 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:11.363062 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:11.363062 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:11.363062 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:11.363062 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:11.363062 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:11.363062 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:11.363062 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:11.363951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:11.363951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:11.363951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:11.363951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:11.363951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:11.363951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:11.363951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:11.363951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:11.363951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:11.363951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:11.363951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:11.363951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:11.363951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:11.363951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:11.363951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:11.363951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:11.364710 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:11.364710 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:11.364710 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:11.364710 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:11.364710 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:11.364710 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:11.364710 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:11.364710 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:11.364710 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:11.364710 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:11.364710 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 03:32:11.364710 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:11.364710 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 49/97] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Oct 17 03:32:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:11.364710 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26980f0d-895d-420f-a3f4-8ea4f4fedefe None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:11.400434 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6801b1-7a0b-4f8d-a560-01e32c60e58d None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:11.434906 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20b0c00-efc6-42ea-a136-3b54fe2f0eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:11.435601 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b0c00-efc6-42ea-a136-3b54fe2f0eb1 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:11.435989 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b0c00-efc6-42ea-a136-3b54fe2f0eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:11.436274 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b0c00-efc6-42ea-a136-3b54fe2f0eb1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:11.446264 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20b0c00-efc6-42ea-a136-3b54fe2f0eb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:11.452133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-b20b0c00-efc6-42ea-a136-3b54fe2f0eb1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 03:32:11.452133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:11.452133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:11.452133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:11.452133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:11.452133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:11.452133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:11.452133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:11.452133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:11.452133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:11.452133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:11.452133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:11.452133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:11.452133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:11.452133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:11.452133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:11.453018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:11.453018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:11.453018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:11.453018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:11.453018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:11.453018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:11.453018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:11.453018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:11.453018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:11.453018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:11.453018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:11.453018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:11.453018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:11.453018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:11.453018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:11.453018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:11.453789 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:11.453789 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:11.453789 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:11.453789 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:11.453789 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:11.453789 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:11.453789 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:11.453789 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:11.453789 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:11.453789 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:11.453789 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 03:32:11.453789 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:11.453789 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 49/98] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:32:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:11.454918 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc9b0c-36ea-4ad5-9f77-1e20761e945c None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:11.454918 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc9b0c-36ea-4ad5-9f77-1e20761e945c None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:11.454918 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbc9b0c-36ea-4ad5-9f77-1e20761e945c None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:11.459854 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 50/99] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:11.459854 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc9b0c-36ea-4ad5-9f77-1e20761e945c None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:11.481307 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26980f0d-895d-420f-a3f4-8ea4f4fedefe None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:11.515307 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085a238-1acb-4a79-b596-e72aee2d3d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:11.515965 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085a238-1acb-4a79-b596-e72aee2d3d49 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:11.516255 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085a238-1acb-4a79-b596-e72aee2d3d49 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:11.516535 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085a238-1acb-4a79-b596-e72aee2d3d49 None admin] PATH_INFO: `/v3/users/6704615fe74b48a4a33d01b3505139b8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:11.517670 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085a238-1acb-4a79-b596-e72aee2d3d49 None admin] RBAC: Authorizing `identity:get_user(user_id=6704615fe74b48a4a33d01b3505139b8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:11.519110 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085a238-1acb-4a79-b596-e72aee2d3d49 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:11.520721 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 50/100] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:32:11 2022] GET /identity/v3/users/6704615fe74b48a4a33d01b3505139b8 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:11.521485 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085a238-1acb-4a79-b596-e72aee2d3d49 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:11.565714 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbc9b0c-36ea-4ad5-9f77-1e20761e945c None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:11.567497 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:11.574331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:11.599933 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0a8da-554b-4703-8951-f99a08fb7dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:11.600544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0a8da-554b-4703-8951-f99a08fb7dc5 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:11.600828 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0a8da-554b-4703-8951-f99a08fb7dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:11.601099 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0a8da-554b-4703-8951-f99a08fb7dc5 None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:11.601753 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0a8da-554b-4703-8951-f99a08fb7dc5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:11.602054 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0a8da-554b-4703-8951-f99a08fb7dc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:11.603506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0a8da-554b-4703-8951-f99a08fb7dc5 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:11.612378 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 51/101] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:32:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:11.613768 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e882769-04c1-4256-bdd3-284a94068123 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:11.614049 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e882769-04c1-4256-bdd3-284a94068123 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:11.614312 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e882769-04c1-4256-bdd3-284a94068123 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:11.615504 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 52/102] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:11.619775 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e882769-04c1-4256-bdd3-284a94068123 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:11.624090 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085a238-1acb-4a79-b596-e72aee2d3d49 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:11.660174 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea60106-7786-4367-9496-d7b5f5a3f788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:11.660806 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea60106-7786-4367-9496-d7b5f5a3f788 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:11.661091 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea60106-7786-4367-9496-d7b5f5a3f788 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:11.661366 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea60106-7786-4367-9496-d7b5f5a3f788 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:11.662468 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea60106-7786-4367-9496-d7b5f5a3f788 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:11.663936 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea60106-7786-4367-9496-d7b5f5a3f788 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:11.665537 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 51/103] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:32:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:11.666629 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587dd702-3d7f-4730-b428-399c5d9da060 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:11.666913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587dd702-3d7f-4730-b428-399c5d9da060 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:11.667186 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587dd702-3d7f-4730-b428-399c5d9da060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:11.684575 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-587dd702-3d7f-4730-b428-399c5d9da060 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:11.686286 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-587dd702-3d7f-4730-b428-399c5d9da060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:11.723927 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e882769-04c1-4256-bdd3-284a94068123 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:11.753589 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 52/104] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:11.754445 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587dd702-3d7f-4730-b428-399c5d9da060 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:11.767377 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051508d5-49d9-4074-b153-34f19f56754e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:11.768084 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051508d5-49d9-4074-b153-34f19f56754e None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:11.768429 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051508d5-49d9-4074-b153-34f19f56754e None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:11.768777 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051508d5-49d9-4074-b153-34f19f56754e None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:11.769506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051508d5-49d9-4074-b153-34f19f56754e None admin] RBAC: Adding query filter params (name=reader) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:11.769876 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051508d5-49d9-4074-b153-34f19f56754e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:11.771343 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051508d5-49d9-4074-b153-34f19f56754e None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:11.784218 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 53/105] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Oct 17 03:32:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:11.784218 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051508d5-49d9-4074-b153-34f19f56754e None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:11.810608 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:11.813441 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:11.860053 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587dd702-3d7f-4730-b428-399c5d9da060 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:11.888001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051508d5-49d9-4074-b153-34f19f56754e None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:11.894595 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc538fe-f688-4441-bb58-afad9d29e277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:11.895211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc538fe-f688-4441-bb58-afad9d29e277 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:11.895500 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc538fe-f688-4441-bb58-afad9d29e277 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:11.895809 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc538fe-f688-4441-bb58-afad9d29e277 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:11.924424 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05a6c2f-f8da-4408-b63b-d8fbc006083a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:11.925326 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a6c2f-f8da-4408-b63b-d8fbc006083a None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:11.925611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a6c2f-f8da-4408-b63b-d8fbc006083a None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:11.925993 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05a6c2f-f8da-4408-b63b-d8fbc006083a None admin] PATH_INFO: `/v3/users/6704615fe74b48a4a33d01b3505139b8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:11.927170 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a6c2f-f8da-4408-b63b-d8fbc006083a None admin] RBAC: Authorizing `identity:get_user(user_id=6704615fe74b48a4a33d01b3505139b8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:11.928641 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05a6c2f-f8da-4408-b63b-d8fbc006083a None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:11.936294 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 54/106] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:32:11 2022] GET /identity/v3/users/6704615fe74b48a4a33d01b3505139b8 => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:11.936294 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699b784-3459-4282-860a-9da68071ed69 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:11.936294 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699b784-3459-4282-860a-9da68071ed69 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:11.936294 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699b784-3459-4282-860a-9da68071ed69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:11.954763 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-9699b784-3459-4282-860a-9da68071ed69 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:11.956069 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-9699b784-3459-4282-860a-9da68071ed69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:11.965770 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc538fe-f688-4441-bb58-afad9d29e277 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:11.967521 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc538fe-f688-4441-bb58-afad9d29e277 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:12.020512 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:12.027788 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 55/107] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:12.028899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9699b784-3459-4282-860a-9da68071ed69 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:12.059425 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:12.075639 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 53/108] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:32:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 03:32:12.075639 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc538fe-f688-4441-bb58-afad9d29e277 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:12.145129 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9699b784-3459-4282-860a-9da68071ed69 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:12.180841 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc538fe-f688-4441-bb58-afad9d29e277 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:12.182842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce7dbe-36f0-404e-843c-0fc7942e1c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:12.184211 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce7dbe-36f0-404e-843c-0fc7942e1c81 None admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:12.184502 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce7dbe-36f0-404e-843c-0fc7942e1c81 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:12.184779 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce7dbe-36f0-404e-843c-0fc7942e1c81 None admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:12.185993 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce7dbe-36f0-404e-843c-0fc7942e1c81 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:12.187442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce7dbe-36f0-404e-843c-0fc7942e1c81 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:12.216789 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284740ef-222f-48b7-b37d-5e11fa3113dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:12.221250 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284740ef-222f-48b7-b37d-5e11fa3113dc None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:12.221250 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284740ef-222f-48b7-b37d-5e11fa3113dc None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:12.221250 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284740ef-222f-48b7-b37d-5e11fa3113dc None admin] PATH_INFO: `/v3/users/9c1e036b000e4fe0b656efea4ab7a75b` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:12.231049 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284740ef-222f-48b7-b37d-5e11fa3113dc None admin] RBAC: Authorizing `identity:get_user(user_id=9c1e036b000e4fe0b656efea4ab7a75b)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:12.233004 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284740ef-222f-48b7-b37d-5e11fa3113dc None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:12.234617 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 54/109] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:32:12 2022] GET /identity/v3/users/9c1e036b000e4fe0b656efea4ab7a75b => generated 289 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:12.236491 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284740ef-222f-48b7-b37d-5e11fa3113dc None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:12.248835 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:12.267466 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 56/110] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Oct 17 03:32:12 2022] POST /identity/v3/projects => generated 302 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:32:12.268343 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce7dbe-36f0-404e-843c-0fc7942e1c81 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:12.359962 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284740ef-222f-48b7-b37d-5e11fa3113dc None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:12.363005 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:12.386860 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce7dbe-36f0-404e-843c-0fc7942e1c81 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:12.399953 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93609149-d219-41c7-a585-9c5576af41b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:12.399953 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93609149-d219-41c7-a585-9c5576af41b9 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:12.399953 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93609149-d219-41c7-a585-9c5576af41b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:12.399953 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93609149-d219-41c7-a585-9c5576af41b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:12.427284 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c0827-90ef-4422-80c9-f55e7fdda822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:12.428221 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c0827-90ef-4422-80c9-f55e7fdda822 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:12.428647 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c0827-90ef-4422-80c9-f55e7fdda822 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:12.429085 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59c0827-90ef-4422-80c9-f55e7fdda822 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:12.440143 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59c0827-90ef-4422-80c9-f55e7fdda822 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:12.445506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-f59c0827-90ef-4422-80c9-f55e7fdda822 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:32:12.445506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:12.445506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:12.445506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:12.445506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:12.445506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:12.445506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:12.445506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:12.445506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:12.445506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:12.445506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:12.445506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:12.445506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:12.445506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:12.445506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:12.445506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:12.446327 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:12.446327 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:12.446327 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:12.446327 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:12.446327 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:12.446327 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:12.446327 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:12.446327 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:12.446327 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:12.446327 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:12.446327 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:12.446327 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:12.446327 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:12.446327 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:12.446327 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:12.446327 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:12.447068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:12.447068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:12.447068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:12.447068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:12.447068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:12.447068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:12.447068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:12.447068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:12.447068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:12.447068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:12.447068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:32:12.447068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:12.447068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 57/111] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Oct 17 03:32:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:12.447674 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c0827-90ef-4422-80c9-f55e7fdda822 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:12.485604 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93609149-d219-41c7-a585-9c5576af41b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:12.487090 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93609149-d219-41c7-a585-9c5576af41b9 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:12.498895 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:12.557169 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59c0827-90ef-4422-80c9-f55e7fdda822 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:12.592867 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 55/112] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:32:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 03:32:12.593728 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93609149-d219-41c7-a585-9c5576af41b9 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:12.598436 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4079d0-013f-4956-8ce9-037b5b37d796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:12.599082 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4079d0-013f-4956-8ce9-037b5b37d796 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:12.599367 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4079d0-013f-4956-8ce9-037b5b37d796 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:12.599641 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4079d0-013f-4956-8ce9-037b5b37d796 None admin] PATH_INFO: `/v3/projects/bc8197db3d20408990645225920f63d7` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:12.600838 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4079d0-013f-4956-8ce9-037b5b37d796 None admin] RBAC: Authorizing `identity:get_project(project_id=bc8197db3d20408990645225920f63d7)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:12.602289 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4079d0-013f-4956-8ce9-037b5b37d796 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:12.603958 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 58/113] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:32:12 2022] GET /identity/v3/projects/bc8197db3d20408990645225920f63d7 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:12.604721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4079d0-013f-4956-8ce9-037b5b37d796 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:12.611866 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:12.705655 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93609149-d219-41c7-a585-9c5576af41b9 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:12.716193 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4079d0-013f-4956-8ce9-037b5b37d796 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:12.717119 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:12.740279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4af23f-030f-46ea-add2-6a84577f31ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:12.740918 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4af23f-030f-46ea-add2-6a84577f31ef None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:12.741218 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4af23f-030f-46ea-add2-6a84577f31ef None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:12.742956 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4af23f-030f-46ea-add2-6a84577f31ef None admin] PATH_INFO: `/v3/projects/bc8197db3d20408990645225920f63d7` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:12.742956 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4af23f-030f-46ea-add2-6a84577f31ef None admin] RBAC: Authorizing `identity:get_project(project_id=bc8197db3d20408990645225920f63d7)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:12.744059 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4af23f-030f-46ea-add2-6a84577f31ef None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:12.745883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 56/114] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:32:12 2022] GET /identity/v3/projects/bc8197db3d20408990645225920f63d7 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:12.746449 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4af23f-030f-46ea-add2-6a84577f31ef None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:12.759092 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8abcd3-cbe0-45a5-8615-90c1ebd3f09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:12.760017 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8abcd3-cbe0-45a5-8615-90c1ebd3f09f None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:12.760438 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8abcd3-cbe0-45a5-8615-90c1ebd3f09f None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:12.760878 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8abcd3-cbe0-45a5-8615-90c1ebd3f09f None admin] PATH_INFO: `/v3/roles/member` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:12.773280 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8abcd3-cbe0-45a5-8615-90c1ebd3f09f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:12.779485 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-6e8abcd3-cbe0-45a5-8615-90c1ebd3f09f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 03:32:12.779485 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:12.779485 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:12.779485 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:12.779485 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:12.779485 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:12.779485 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:12.779485 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:12.779485 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:12.779485 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:12.779485 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:12.779485 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:12.779485 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:12.779485 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:12.779485 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:12.779485 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:12.780359 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:12.780359 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:12.780359 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:12.780359 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:12.780359 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:12.780359 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:12.780359 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:12.780359 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:12.780359 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:12.780359 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:12.780359 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:12.780359 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:12.780359 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:12.780359 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:12.780359 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:12.780359 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:12.781083 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:12.781083 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:12.781083 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:12.781083 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:12.781083 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:12.781083 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:12.781083 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:12.781083 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:12.781083 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:12.781083 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:12.781083 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 03:32:12.781083 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:12.781083 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 59/115] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:32:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:12.781719 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8abcd3-cbe0-45a5-8615-90c1ebd3f09f None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:12.852329 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:12.852799 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4af23f-030f-46ea-add2-6a84577f31ef None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:12.885966 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87864869-f77b-49cb-9d6d-742097f62dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:12.886564 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87864869-f77b-49cb-9d6d-742097f62dc4 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:12.886856 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87864869-f77b-49cb-9d6d-742097f62dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:12.887120 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87864869-f77b-49cb-9d6d-742097f62dc4 None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:12.887817 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87864869-f77b-49cb-9d6d-742097f62dc4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:12.888124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87864869-f77b-49cb-9d6d-742097f62dc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:12.890000 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87864869-f77b-49cb-9d6d-742097f62dc4 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:12.896120 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8abcd3-cbe0-45a5-8615-90c1ebd3f09f None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:12.898395 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 57/116] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:32:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:12.899156 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87864869-f77b-49cb-9d6d-742097f62dc4 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:12.939452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73358d-0022-46d0-b1ba-430fd0c3b723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:12.940138 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73358d-0022-46d0-b1ba-430fd0c3b723 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:12.940486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73358d-0022-46d0-b1ba-430fd0c3b723 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:12.940769 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73358d-0022-46d0-b1ba-430fd0c3b723 None admin] PATH_INFO: `/v3/projects/bc8197db3d20408990645225920f63d7` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:12.941512 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:12.942472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73358d-0022-46d0-b1ba-430fd0c3b723 None admin] RBAC: Authorizing `identity:get_project(project_id=bc8197db3d20408990645225920f63d7)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:12.944012 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73358d-0022-46d0-b1ba-430fd0c3b723 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:12.950484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 60/117] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:32:12 2022] GET /identity/v3/projects/bc8197db3d20408990645225920f63d7 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:12.950484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73358d-0022-46d0-b1ba-430fd0c3b723 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:13.006171 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87864869-f77b-49cb-9d6d-742097f62dc4 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:13.042016 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b61abd-2437-4517-825b-619718acd746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:13.042678 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b61abd-2437-4517-825b-619718acd746 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:13.043043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b61abd-2437-4517-825b-619718acd746 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:13.043323 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b61abd-2437-4517-825b-619718acd746 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:13.044084 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b61abd-2437-4517-825b-619718acd746 None admin] RBAC: Adding query filter params (role.id=84a147decc7b417783086c1effe0fe20, scope.project.id=bc8197db3d20408990645225920f63d7, user.id=6704615fe74b48a4a33d01b3505139b8) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:13.044413 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b61abd-2437-4517-825b-619718acd746 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:13.045861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b61abd-2437-4517-825b-619718acd746 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:13.059763 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 58/118] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Oct 17 03:32:12 2022] GET /identity/v3/role_assignments?user.id=6704615fe74b48a4a33d01b3505139b8&scope.project.id=bc8197db3d20408990645225920f63d7&role.id=84a147decc7b417783086c1effe0fe20 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:13.059763 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b61abd-2437-4517-825b-619718acd746 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:13.072259 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73358d-0022-46d0-b1ba-430fd0c3b723 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:13.109944 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:13.115492 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce786180-82e4-4804-a6cb-83f1d22c803b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:13.120625 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce786180-82e4-4804-a6cb-83f1d22c803b None admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:13.120625 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce786180-82e4-4804-a6cb-83f1d22c803b None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:13.120625 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce786180-82e4-4804-a6cb-83f1d22c803b None admin] PATH_INFO: `/v3/projects/bc8197db3d20408990645225920f63d7/users/9c1e036b000e4fe0b656efea4ab7a75b/roles/010a82aa9f3b4a23817fe234196cf259` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:13.120625 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce786180-82e4-4804-a6cb-83f1d22c803b None admin] RBAC: Authorizing `identity:create_grant(project_id=bc8197db3d20408990645225920f63d7, user_id=9c1e036b000e4fe0b656efea4ab7a75b, role_id=010a82aa9f3b4a23817fe234196cf259)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:13.121356 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce786180-82e4-4804-a6cb-83f1d22c803b None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:13.183510 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b61abd-2437-4517-825b-619718acd746 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:13.190943 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:13.194111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 61/119] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Oct 17 03:32:12 2022] PUT /identity/v3/projects/bc8197db3d20408990645225920f63d7/users/9c1e036b000e4fe0b656efea4ab7a75b/roles/010a82aa9f3b4a23817fe234196cf259 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:32:13.194993 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce786180-82e4-4804-a6cb-83f1d22c803b None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:13.257858 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b067c7f-4197-480b-869b-a7f763816cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:13.258499 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b067c7f-4197-480b-869b-a7f763816cab None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:13.258787 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b067c7f-4197-480b-869b-a7f763816cab None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:13.259067 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b067c7f-4197-480b-869b-a7f763816cab None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:13.260955 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b067c7f-4197-480b-869b-a7f763816cab None admin] RBAC: Adding query filter params (name=member) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:13.261272 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b067c7f-4197-480b-869b-a7f763816cab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:13.266245 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b067c7f-4197-480b-869b-a7f763816cab None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:13.274375 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 59/120] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Oct 17 03:32:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:13.275680 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b067c7f-4197-480b-869b-a7f763816cab None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:13.346948 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce786180-82e4-4804-a6cb-83f1d22c803b None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:13.395893 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d045bb-36ed-4e22-8386-436e7cee7696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:13.396552 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d045bb-36ed-4e22-8386-436e7cee7696 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:13.396831 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d045bb-36ed-4e22-8386-436e7cee7696 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:13.397171 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d045bb-36ed-4e22-8386-436e7cee7696 None admin] PATH_INFO: `/v3/users/f09f98306cd54b068020428d74bd0957` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:13.398347 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d045bb-36ed-4e22-8386-436e7cee7696 None admin] RBAC: Authorizing `identity:get_user(user_id=f09f98306cd54b068020428d74bd0957)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:13.398655 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b067c7f-4197-480b-869b-a7f763816cab None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:13.399860 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d045bb-36ed-4e22-8386-436e7cee7696 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:13.401611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 62/121] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:32:13 2022] GET /identity/v3/users/f09f98306cd54b068020428d74bd0957 => generated 247 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:13.402459 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d045bb-36ed-4e22-8386-436e7cee7696 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:13.443590 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c5f2de-27a5-4ea1-867f-0894cbedb8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:13.444317 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5f2de-27a5-4ea1-867f-0894cbedb8f2 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:13.444741 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5f2de-27a5-4ea1-867f-0894cbedb8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:13.445023 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5f2de-27a5-4ea1-867f-0894cbedb8f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:13.446581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c5f2de-27a5-4ea1-867f-0894cbedb8f2 None admin] RBAC: Adding query filter params (role.id=609a4bf154ff4dcabb408b3789d28983, scope.project.id=bc8197db3d20408990645225920f63d7, user.id=6704615fe74b48a4a33d01b3505139b8) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:13.446906 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c5f2de-27a5-4ea1-867f-0894cbedb8f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:13.448408 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c5f2de-27a5-4ea1-867f-0894cbedb8f2 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:13.455528 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 60/122] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Oct 17 03:32:13 2022] GET /identity/v3/role_assignments?user.id=6704615fe74b48a4a33d01b3505139b8&scope.project.id=bc8197db3d20408990645225920f63d7&role.id=609a4bf154ff4dcabb408b3789d28983 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:13.456132 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c5f2de-27a5-4ea1-867f-0894cbedb8f2 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:13.490074 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:13.502507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:13.573520 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d045bb-36ed-4e22-8386-436e7cee7696 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:13.586726 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c5f2de-27a5-4ea1-867f-0894cbedb8f2 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:13.636698 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc9a86-8ab5-4f9f-b121-c29aaf5ba274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:13.637347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc9a86-8ab5-4f9f-b121-c29aaf5ba274 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:13.637630 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc9a86-8ab5-4f9f-b121-c29aaf5ba274 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:13.638003 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc9a86-8ab5-4f9f-b121-c29aaf5ba274 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:13.659834 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e73fb14-07c1-4918-8fc6-aba55302a9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:13.660535 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73fb14-07c1-4918-8fc6-aba55302a9f5 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:13.660818 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73fb14-07c1-4918-8fc6-aba55302a9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:13.661092 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e73fb14-07c1-4918-8fc6-aba55302a9f5 None admin] PATH_INFO: `/v3/users/6704615fe74b48a4a33d01b3505139b8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:13.662694 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e73fb14-07c1-4918-8fc6-aba55302a9f5 None admin] RBAC: Authorizing `identity:get_user(user_id=6704615fe74b48a4a33d01b3505139b8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:13.664219 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e73fb14-07c1-4918-8fc6-aba55302a9f5 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:13.666623 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 63/123] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:32:13 2022] GET /identity/v3/users/6704615fe74b48a4a33d01b3505139b8 => generated 275 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:13.673448 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e73fb14-07c1-4918-8fc6-aba55302a9f5 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:13.732528 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc9a86-8ab5-4f9f-b121-c29aaf5ba274 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:13.734375 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc9a86-8ab5-4f9f-b121-c29aaf5ba274 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:13.767695 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:13.803720 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e73fb14-07c1-4918-8fc6-aba55302a9f5 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:13.837745 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:13.846380 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92616c9-cb8d-473b-b017-abe691e3aefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:13.846999 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92616c9-cb8d-473b-b017-abe691e3aefc None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:13.847278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92616c9-cb8d-473b-b017-abe691e3aefc None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:13.847552 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92616c9-cb8d-473b-b017-abe691e3aefc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:13.849194 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 61/124] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:32:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 03:32:13.855439 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc9a86-8ab5-4f9f-b121-c29aaf5ba274 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:13.937680 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92616c9-cb8d-473b-b017-abe691e3aefc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:13.939131 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92616c9-cb8d-473b-b017-abe691e3aefc None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:13.998033 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc9a86-8ab5-4f9f-b121-c29aaf5ba274 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:14.037041 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:14.043122 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 64/125] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:32:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 03:32:14.049139 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411507ba-c584-4670-85f0-1e5b5f1d642b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:14.052039 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92616c9-cb8d-473b-b017-abe691e3aefc None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:14.052147 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411507ba-c584-4670-85f0-1e5b5f1d642b None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:14.052147 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411507ba-c584-4670-85f0-1e5b5f1d642b None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:14.052147 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411507ba-c584-4670-85f0-1e5b5f1d642b None admin] PATH_INFO: `/v3/projects/bc8197db3d20408990645225920f63d7` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:14.052147 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411507ba-c584-4670-85f0-1e5b5f1d642b None admin] RBAC: Authorizing `identity:get_project(project_id=bc8197db3d20408990645225920f63d7)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:14.054375 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411507ba-c584-4670-85f0-1e5b5f1d642b None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:14.056130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 62/126] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:32:13 2022] GET /identity/v3/projects/bc8197db3d20408990645225920f63d7 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:14.062090 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411507ba-c584-4670-85f0-1e5b5f1d642b None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:14.083546 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:14.175728 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411507ba-c584-4670-85f0-1e5b5f1d642b None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:14.180462 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92616c9-cb8d-473b-b017-abe691e3aefc None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:14.211131 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6629558-d0a4-4727-bf37-63cabd9d95e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:14.211995 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6629558-d0a4-4727-bf37-63cabd9d95e8 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:14.212419 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6629558-d0a4-4727-bf37-63cabd9d95e8 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:14.212887 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6629558-d0a4-4727-bf37-63cabd9d95e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:14.213817 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6629558-d0a4-4727-bf37-63cabd9d95e8 None admin] RBAC: Adding query filter params (role.id=611120afb3a544d0b5161da1b769a2f8, scope.project.id=bc8197db3d20408990645225920f63d7, user.id=f09f98306cd54b068020428d74bd0957) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:14.214232 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6629558-d0a4-4727-bf37-63cabd9d95e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:14.216155 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6629558-d0a4-4727-bf37-63cabd9d95e8 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:14.223080 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f99a1-3baa-4f9d-abed-6e9c9fe16f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:14.224066 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 63/127] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Oct 17 03:32:14 2022] GET /identity/v3/role_assignments?user.id=f09f98306cd54b068020428d74bd0957&scope.project.id=bc8197db3d20408990645225920f63d7&role.id=611120afb3a544d0b5161da1b769a2f8 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:14.224235 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f99a1-3baa-4f9d-abed-6e9c9fe16f27 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:14.224235 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f99a1-3baa-4f9d-abed-6e9c9fe16f27 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:14.224564 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f99a1-3baa-4f9d-abed-6e9c9fe16f27 None admin] PATH_INFO: `/v3/projects/b423bebfa6774da6a57f373ee36fbebb` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:14.229262 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f99a1-3baa-4f9d-abed-6e9c9fe16f27 None admin] RBAC: Authorizing `identity:get_project(project_id=b423bebfa6774da6a57f373ee36fbebb)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:14.230095 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f99a1-3baa-4f9d-abed-6e9c9fe16f27 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:14.232775 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 65/128] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:32:14 2022] GET /identity/v3/projects/b423bebfa6774da6a57f373ee36fbebb => generated 312 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:14.238431 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6629558-d0a4-4727-bf37-63cabd9d95e8 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:14.314646 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:14.358736 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6629558-d0a4-4727-bf37-63cabd9d95e8 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:14.397171 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e88f3-814d-4f53-9fe5-1f65efc84c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:14.397799 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e88f3-814d-4f53-9fe5-1f65efc84c96 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:14.398145 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e88f3-814d-4f53-9fe5-1f65efc84c96 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:14.398422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e88f3-814d-4f53-9fe5-1f65efc84c96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:14.399218 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e88f3-814d-4f53-9fe5-1f65efc84c96 None admin] RBAC: Adding query filter params (role.id=84a147decc7b417783086c1effe0fe20, scope.project.id=b423bebfa6774da6a57f373ee36fbebb, user.id=6704615fe74b48a4a33d01b3505139b8) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:14.399533 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e88f3-814d-4f53-9fe5-1f65efc84c96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:14.401067 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e88f3-814d-4f53-9fe5-1f65efc84c96 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:14.408988 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 64/129] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Oct 17 03:32:14 2022] GET /identity/v3/role_assignments?user.id=6704615fe74b48a4a33d01b3505139b8&scope.project.id=b423bebfa6774da6a57f373ee36fbebb&role.id=84a147decc7b417783086c1effe0fe20 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:14.599100 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64a5a9-5348-48bd-9099-611ce32d2010 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:14.599315 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64a5a9-5348-48bd-9099-611ce32d2010 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:14.599595 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64a5a9-5348-48bd-9099-611ce32d2010 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:14.601024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 66/130] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:14.606299 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560629e-b38f-4d19-a1bf-aa6196d4597e None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:14.606598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560629e-b38f-4d19-a1bf-aa6196d4597e None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:14.606921 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560629e-b38f-4d19-a1bf-aa6196d4597e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:14.627119 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-1560629e-b38f-4d19-a1bf-aa6196d4597e None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:14.628413 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-1560629e-b38f-4d19-a1bf-aa6196d4597e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:14.704099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 65/131] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:14.967243 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64a5a9-5348-48bd-9099-611ce32d2010 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:15.057927 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:15.064262 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64a5a9-5348-48bd-9099-611ce32d2010 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:15.098517 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35966537-52c0-4042-9d7b-ec04b632b7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:15.099154 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35966537-52c0-4042-9d7b-ec04b632b7a8 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:15.099438 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35966537-52c0-4042-9d7b-ec04b632b7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:15.099715 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35966537-52c0-4042-9d7b-ec04b632b7a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:15.100902 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35966537-52c0-4042-9d7b-ec04b632b7a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:15.102345 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35966537-52c0-4042-9d7b-ec04b632b7a8 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:15.103965 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 67/132] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:32:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:15.109408 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1560629e-b38f-4d19-a1bf-aa6196d4597e None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:15.163980 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:15.208016 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1560629e-b38f-4d19-a1bf-aa6196d4597e None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:15.241567 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096a00d-92d0-4e02-93d3-d8f2601e5618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:15.242123 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096a00d-92d0-4e02-93d3-d8f2601e5618 None admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:15.242411 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096a00d-92d0-4e02-93d3-d8f2601e5618 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:15.242693 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096a00d-92d0-4e02-93d3-d8f2601e5618 None admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:15.243365 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096a00d-92d0-4e02-93d3-d8f2601e5618 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:15.244846 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096a00d-92d0-4e02-93d3-d8f2601e5618 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:15.338797 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 66/133] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Oct 17 03:32:15 2022] POST /identity/v3/users => generated 283 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:32:15.558514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b95b00-f9ba-4f1b-bff5-f08aff536906 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:15.601940 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b95b00-f9ba-4f1b-bff5-f08aff536906 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:15.601940 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b95b00-f9ba-4f1b-bff5-f08aff536906 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:15.601940 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 68/134] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:15.602402 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1aeb5-865c-4f0d-b1db-21d8926b1105 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:15.602402 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1aeb5-865c-4f0d-b1db-21d8926b1105 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:15.602402 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1aeb5-865c-4f0d-b1db-21d8926b1105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:15.602402 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-a1c1aeb5-865c-4f0d-b1db-21d8926b1105 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:15.602402 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-a1c1aeb5-865c-4f0d-b1db-21d8926b1105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:15.658291 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 67/135] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:15.929361 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b95b00-f9ba-4f1b-bff5-f08aff536906 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:16.032035 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b95b00-f9ba-4f1b-bff5-f08aff536906 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:16.066697 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee122f1-ffbe-4000-ab98-33ad56960265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:16.067345 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee122f1-ffbe-4000-ab98-33ad56960265 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:16.067622 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee122f1-ffbe-4000-ab98-33ad56960265 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:16.067925 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee122f1-ffbe-4000-ab98-33ad56960265 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:16.078402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee122f1-ffbe-4000-ab98-33ad56960265 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:16.083369 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-2ee122f1-ffbe-4000-ab98-33ad56960265 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 03:32:16.083369 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:16.083369 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:16.083369 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:16.083369 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:16.083369 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:16.083369 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:16.083369 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:16.083369 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:16.083369 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:16.083369 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:16.083369 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:16.083369 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:16.083369 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:16.083369 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:16.083369 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:16.084274 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:16.084274 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:16.084274 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:16.084274 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:16.084274 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:16.084274 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:16.084274 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:16.084274 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:16.084274 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:16.084274 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:16.084274 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:16.084274 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:16.084274 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:16.084274 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:16.084274 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:16.084274 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:16.085032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:16.085032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:16.085032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:16.085032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:16.085032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:16.085032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:16.085032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:16.085032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:16.085032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:16.085032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:16.085032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 03:32:16.085032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:16.085032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 69/136] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:32:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:16.092633 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c1aeb5-865c-4f0d-b1db-21d8926b1105 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:16.136250 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:16.195052 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c1aeb5-865c-4f0d-b1db-21d8926b1105 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:16.231140 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b2c8a-39e9-4711-a794-83b7b23eca86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:16.232808 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b2c8a-39e9-4711-a794-83b7b23eca86 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:16.233278 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b2c8a-39e9-4711-a794-83b7b23eca86 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:16.233728 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2b2c8a-39e9-4711-a794-83b7b23eca86 None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:16.234576 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b2c8a-39e9-4711-a794-83b7b23eca86 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:16.235057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b2c8a-39e9-4711-a794-83b7b23eca86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:16.236656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2b2c8a-39e9-4711-a794-83b7b23eca86 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:16.245503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 68/137] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Oct 17 03:32:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:16.256988 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee122f1-ffbe-4000-ab98-33ad56960265 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:16.282972 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:16.363114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee122f1-ffbe-4000-ab98-33ad56960265 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:16.397205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab65378-05a6-40d4-b63e-f2ad6815aae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:16.397842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab65378-05a6-40d4-b63e-f2ad6815aae3 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:16.398120 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab65378-05a6-40d4-b63e-f2ad6815aae3 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:16.398389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab65378-05a6-40d4-b63e-f2ad6815aae3 None admin] PATH_INFO: `/v3/users/9c1e036b000e4fe0b656efea4ab7a75b` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:16.399528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab65378-05a6-40d4-b63e-f2ad6815aae3 None admin] RBAC: Authorizing `identity:get_user(user_id=9c1e036b000e4fe0b656efea4ab7a75b)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:16.401108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab65378-05a6-40d4-b63e-f2ad6815aae3 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:16.402867 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 70/138] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:32:16 2022] GET /identity/v3/users/9c1e036b000e4fe0b656efea4ab7a75b => generated 289 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:16.408889 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b2c8a-39e9-4711-a794-83b7b23eca86 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:16.512756 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2b2c8a-39e9-4711-a794-83b7b23eca86 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:16.516500 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:16.550949 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c5250-1aed-47b4-b4ec-b1083eb68d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:16.551921 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c5250-1aed-47b4-b4ec-b1083eb68d99 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:16.552385 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c5250-1aed-47b4-b4ec-b1083eb68d99 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:16.552839 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c5250-1aed-47b4-b4ec-b1083eb68d99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:16.625833 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c5250-1aed-47b4-b4ec-b1083eb68d99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:16.627308 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c5250-1aed-47b4-b4ec-b1083eb68d99 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:16.729713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 69/139] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:32:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 03:32:16.735802 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab65378-05a6-40d4-b63e-f2ad6815aae3 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:16.826670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:16.835438 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab65378-05a6-40d4-b63e-f2ad6815aae3 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:16.869733 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedd3219-0896-4781-8d4e-5f119c53b001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:16.870668 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd3219-0896-4781-8d4e-5f119c53b001 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:16.871152 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd3219-0896-4781-8d4e-5f119c53b001 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:16.871600 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd3219-0896-4781-8d4e-5f119c53b001 None admin] PATH_INFO: `/v3/projects/bc8197db3d20408990645225920f63d7` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:16.873426 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd3219-0896-4781-8d4e-5f119c53b001 None admin] RBAC: Authorizing `identity:get_project(project_id=bc8197db3d20408990645225920f63d7)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:16.875029 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedd3219-0896-4781-8d4e-5f119c53b001 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:16.877164 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 71/140] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:32:16 2022] GET /identity/v3/projects/bc8197db3d20408990645225920f63d7 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:16.885042 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c5250-1aed-47b4-b4ec-b1083eb68d99 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:16.893862 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:16.994212 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c5250-1aed-47b4-b4ec-b1083eb68d99 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:17.030967 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5851cf3f-1534-4b70-aec0-418ed63874d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:17.031946 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851cf3f-1534-4b70-aec0-418ed63874d7 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:17.032487 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851cf3f-1534-4b70-aec0-418ed63874d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:17.032959 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851cf3f-1534-4b70-aec0-418ed63874d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:17.033927 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851cf3f-1534-4b70-aec0-418ed63874d7 None admin] RBAC: Adding query filter params (role.id=010a82aa9f3b4a23817fe234196cf259, scope.project.id=bc8197db3d20408990645225920f63d7, user.id=9c1e036b000e4fe0b656efea4ab7a75b) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:17.034503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851cf3f-1534-4b70-aec0-418ed63874d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:17.036197 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851cf3f-1534-4b70-aec0-418ed63874d7 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:17.047269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 70/141] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Oct 17 03:32:16 2022] GET /identity/v3/role_assignments?user.id=9c1e036b000e4fe0b656efea4ab7a75b&scope.project.id=bc8197db3d20408990645225920f63d7&role.id=010a82aa9f3b4a23817fe234196cf259 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:17.586896 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355adb8-c5f4-4769-8ab1-79bb5c568919 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:17.587263 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355adb8-c5f4-4769-8ab1-79bb5c568919 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:17.587456 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6355adb8-c5f4-4769-8ab1-79bb5c568919 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:17.588851 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 72/142] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:32:17.594713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d8ca4-05ab-42e4-ae4a-c29e8fc5258a None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:17.595139 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d8ca4-05ab-42e4-ae4a-c29e8fc5258a None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:17.595341 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d8ca4-05ab-42e4-ae4a-c29e8fc5258a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:17.614113 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-ad9d8ca4-05ab-42e4-ae4a-c29e8fc5258a None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:17.615293 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-ad9d8ca4-05ab-42e4-ae4a-c29e8fc5258a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:17.682165 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:17.687773 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 71/143] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:17.834078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe8567-170f-448e-945e-639cf85a064a None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:17.834582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe8567-170f-448e-945e-639cf85a064a None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:17.834698 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe8567-170f-448e-945e-639cf85a064a None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:17.836159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 73/144] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:17.841872 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53984e5-21b4-4b90-99ae-e4f75dbb66e7 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:17.842107 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53984e5-21b4-4b90-99ae-e4f75dbb66e7 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:17.842386 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53984e5-21b4-4b90-99ae-e4f75dbb66e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:17.861971 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-b53984e5-21b4-4b90-99ae-e4f75dbb66e7 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:17.863158 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-b53984e5-21b4-4b90-99ae-e4f75dbb66e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:17.886955 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18086367-49a4-40b7-a20f-1ed793bc1783 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:17.887239 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18086367-49a4-40b7-a20f-1ed793bc1783 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:17.887818 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18086367-49a4-40b7-a20f-1ed793bc1783 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:17.888920 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 74/145] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:32:17.893590 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a5896-9308-433b-b52e-e97cb7dd1c43 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:17.893876 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a5896-9308-433b-b52e-e97cb7dd1c43 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:17.894155 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a5896-9308-433b-b52e-e97cb7dd1c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:17.913002 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-e83a5896-9308-433b-b52e-e97cb7dd1c43 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:17.914201 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-e83a5896-9308-433b-b52e-e97cb7dd1c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:17.934572 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 72/146] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:17.952721 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53984e5-21b4-4b90-99ae-e4f75dbb66e7 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:17.985750 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 75/147] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 03:32:18.051695 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53984e5-21b4-4b90-99ae-e4f75dbb66e7 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:18.077724 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:18.085699 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0214cc-804b-4a70-896f-3cbc0f0871c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:18.088632 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0214cc-804b-4a70-896f-3cbc0f0871c2 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:18.088632 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0214cc-804b-4a70-896f-3cbc0f0871c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:18.088632 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0214cc-804b-4a70-896f-3cbc0f0871c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:18.098171 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0214cc-804b-4a70-896f-3cbc0f0871c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:18.103428 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-6e0214cc-804b-4a70-896f-3cbc0f0871c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:32:18.103428 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:18.103428 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:18.103428 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:18.103428 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:18.103428 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:18.103428 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:18.103428 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:18.103428 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:18.103428 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:18.103428 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:18.103428 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:18.103428 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:18.103428 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:18.103428 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:18.103428 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:18.104714 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:18.104714 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:18.104714 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:18.104714 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:18.104714 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:18.104714 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:18.104714 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:18.104714 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:18.104714 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:18.104714 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:18.104714 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:18.104714 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:18.104714 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:18.104714 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:18.104714 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:18.104714 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:18.105532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:18.105532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:18.105532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:18.105532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:18.105532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:18.105532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:18.105532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:18.105532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:18.105532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:18.105532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:18.105532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:32:18.105532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:18.105532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 73/148] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Oct 17 03:32:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:18.115954 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a5896-9308-433b-b52e-e97cb7dd1c43 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:18.201024 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0214cc-804b-4a70-896f-3cbc0f0871c2 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:18.205694 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:18.216861 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a5896-9308-433b-b52e-e97cb7dd1c43 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:18.258909 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155e21f-b41c-4891-aa5f-062b95b85a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:18.259530 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155e21f-b41c-4891-aa5f-062b95b85a0e None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:18.259844 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155e21f-b41c-4891-aa5f-062b95b85a0e None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:18.260120 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155e21f-b41c-4891-aa5f-062b95b85a0e None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:18.260849 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9155e21f-b41c-4891-aa5f-062b95b85a0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:18.261210 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9155e21f-b41c-4891-aa5f-062b95b85a0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:18.262759 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9155e21f-b41c-4891-aa5f-062b95b85a0e None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:18.275851 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 76/149] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:32:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:18.275851 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155e21f-b41c-4891-aa5f-062b95b85a0e None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:18.305665 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0214cc-804b-4a70-896f-3cbc0f0871c2 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:18.338507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d5353-5738-4332-b609-546e3bea6206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:18.339142 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d5353-5738-4332-b609-546e3bea6206 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:18.339419 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d5353-5738-4332-b609-546e3bea6206 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:18.339691 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d5353-5738-4332-b609-546e3bea6206 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:18.350739 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80d5353-5738-4332-b609-546e3bea6206 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:18.355520 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-c80d5353-5738-4332-b609-546e3bea6206 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 03:32:18.355520 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:18.355520 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:18.355520 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:18.355520 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:18.355520 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:18.355520 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:18.355520 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:18.355520 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:18.355520 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:18.355520 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:18.355520 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:18.355520 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:18.355520 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:18.355520 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:18.355520 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:18.356503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:18.356503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:18.356503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:18.356503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:18.356503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:18.356503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:18.356503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:18.356503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:18.356503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:18.356503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:18.356503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:18.356503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:18.356503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:18.356503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:18.356503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:18.356503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:18.357318 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:18.357318 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:18.357318 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:18.357318 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:18.357318 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:18.357318 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:18.357318 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:18.357318 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:18.357318 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:18.357318 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:18.357318 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 03:32:18.357318 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:18.357318 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 74/150] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Oct 17 03:32:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:32:18.358223 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d5353-5738-4332-b609-546e3bea6206 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:18.377248 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155e21f-b41c-4891-aa5f-062b95b85a0e None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:18.405804 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:18.410420 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16121890-d0ff-45cc-87e6-559494415258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:18.414598 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16121890-d0ff-45cc-87e6-559494415258 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:18.414598 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16121890-d0ff-45cc-87e6-559494415258 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:18.414598 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16121890-d0ff-45cc-87e6-559494415258 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:18.414598 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16121890-d0ff-45cc-87e6-559494415258 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:18.414598 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16121890-d0ff-45cc-87e6-559494415258 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:18.415763 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 77/151] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:32:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:18.416529 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16121890-d0ff-45cc-87e6-559494415258 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:18.437765 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:18.459082 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d5353-5738-4332-b609-546e3bea6206 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:18.491304 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbcac6-c5f9-49ae-aeae-7cd036996f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:18.491958 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbcac6-c5f9-49ae-aeae-7cd036996f0e None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:18.492240 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbcac6-c5f9-49ae-aeae-7cd036996f0e None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:18.492509 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cbcac6-c5f9-49ae-aeae-7cd036996f0e None admin] PATH_INFO: `/v3/users/8a4e5ca94a4d4e3fa0026821ce93f562` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:18.505945 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cbcac6-c5f9-49ae-aeae-7cd036996f0e None admin] RBAC: Authorizing `identity:get_user(user_id=8a4e5ca94a4d4e3fa0026821ce93f562)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:18.507396 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cbcac6-c5f9-49ae-aeae-7cd036996f0e None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:18.509039 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 75/152] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:32:18 2022] GET /identity/v3/users/8a4e5ca94a4d4e3fa0026821ce93f562 => generated 283 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:18.509782 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbcac6-c5f9-49ae-aeae-7cd036996f0e None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:18.512408 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16121890-d0ff-45cc-87e6-559494415258 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:18.544966 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f9719-53d1-454e-9ee8-982862970187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:18.545580 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f9719-53d1-454e-9ee8-982862970187 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:18.545859 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f9719-53d1-454e-9ee8-982862970187 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:18.546126 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f9719-53d1-454e-9ee8-982862970187 None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:18.546779 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f9719-53d1-454e-9ee8-982862970187 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:18.547083 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f9719-53d1-454e-9ee8-982862970187 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:18.548443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f9719-53d1-454e-9ee8-982862970187 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:18.556442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 78/153] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:32:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:18.562767 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f9719-53d1-454e-9ee8-982862970187 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:18.606318 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cbcac6-c5f9-49ae-aeae-7cd036996f0e None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:18.613442 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:18.640938 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41c3a2-7e6b-4d06-9ee1-0561f458fdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:18.641545 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41c3a2-7e6b-4d06-9ee1-0561f458fdb9 None admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:18.641834 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41c3a2-7e6b-4d06-9ee1-0561f458fdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:18.642108 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41c3a2-7e6b-4d06-9ee1-0561f458fdb9 None admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:18.642758 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41c3a2-7e6b-4d06-9ee1-0561f458fdb9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:18.643104 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:18.644279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a41c3a2-7e6b-4d06-9ee1-0561f458fdb9 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:18.660457 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f9719-53d1-454e-9ee8-982862970187 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:18.693754 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad47f1-49e5-466a-af5c-af5457688bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:18.695068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad47f1-49e5-466a-af5c-af5457688bce None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:18.695068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad47f1-49e5-466a-af5c-af5457688bce None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:18.695068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad47f1-49e5-466a-af5c-af5457688bce None admin] PATH_INFO: `/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:18.696027 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad47f1-49e5-466a-af5c-af5457688bce None admin] RBAC: Authorizing `identity:get_project(project_id=9a1236f1bb0f4a499c0206128e5c97f9)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:18.697435 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad47f1-49e5-466a-af5c-af5457688bce None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:18.699052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 79/154] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:32:18 2022] GET /identity/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9 => generated 302 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:18.699052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad47f1-49e5-466a-af5c-af5457688bce None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:18.746085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 76/155] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Oct 17 03:32:18 2022] POST /identity/v3/users => generated 297 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:32:18.746874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41c3a2-7e6b-4d06-9ee1-0561f458fdb9 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:18.812355 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad47f1-49e5-466a-af5c-af5457688bce None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:18.849104 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f50ce-9431-4082-b9a9-74ea796969be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:18.849752 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f50ce-9431-4082-b9a9-74ea796969be None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:18.850030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f50ce-9431-4082-b9a9-74ea796969be None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:18.850308 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f50ce-9431-4082-b9a9-74ea796969be None admin] PATH_INFO: `/v3/users/8a4e5ca94a4d4e3fa0026821ce93f562` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:18.851499 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6f50ce-9431-4082-b9a9-74ea796969be None admin] RBAC: Authorizing `identity:get_user(user_id=8a4e5ca94a4d4e3fa0026821ce93f562)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:18.853010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6f50ce-9431-4082-b9a9-74ea796969be None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:18.859775 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 80/156] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:32:18 2022] GET /identity/v3/users/8a4e5ca94a4d4e3fa0026821ce93f562 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:18.860888 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f50ce-9431-4082-b9a9-74ea796969be None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:18.880835 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41c3a2-7e6b-4d06-9ee1-0561f458fdb9 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:18.882260 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:18.918672 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da2fa8-5bfb-4569-b60b-36073216c214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:18.918942 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da2fa8-5bfb-4569-b60b-36073216c214 None admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:18.919720 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da2fa8-5bfb-4569-b60b-36073216c214 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:18.920036 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66da2fa8-5bfb-4569-b60b-36073216c214 None admin] PATH_INFO: `/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9/users/8a4e5ca94a4d4e3fa0026821ce93f562/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:18.921994 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da2fa8-5bfb-4569-b60b-36073216c214 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a1236f1bb0f4a499c0206128e5c97f9, user_id=8a4e5ca94a4d4e3fa0026821ce93f562, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:18.923442 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66da2fa8-5bfb-4569-b60b-36073216c214 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:18.975301 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f50ce-9431-4082-b9a9-74ea796969be None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:19.004732 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 77/157] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Oct 17 03:32:18 2022] PUT /identity/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9/users/8a4e5ca94a4d4e3fa0026821ce93f562/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:32:19.011180 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb78fc-f4c7-42cb-928e-76bcbe2f5b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:19.011982 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb78fc-f4c7-42cb-928e-76bcbe2f5b6f None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:19.012270 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb78fc-f4c7-42cb-928e-76bcbe2f5b6f None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:19.016149 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb78fc-f4c7-42cb-928e-76bcbe2f5b6f None admin] PATH_INFO: `/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:19.016149 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb78fc-f4c7-42cb-928e-76bcbe2f5b6f None admin] RBAC: Authorizing `identity:get_project(project_id=9a1236f1bb0f4a499c0206128e5c97f9)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:19.016149 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb78fc-f4c7-42cb-928e-76bcbe2f5b6f None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:19.017792 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 81/158] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:32:18 2022] GET /identity/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:19.023529 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da2fa8-5bfb-4569-b60b-36073216c214 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:19.053918 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:19.152889 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66da2fa8-5bfb-4569-b60b-36073216c214 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:19.193602 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3319688d-6b48-48a0-bd14-defafcd5f2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:19.194359 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319688d-6b48-48a0-bd14-defafcd5f2b9 None admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:19.194640 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319688d-6b48-48a0-bd14-defafcd5f2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:19.194921 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319688d-6b48-48a0-bd14-defafcd5f2b9 None admin] PATH_INFO: `/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9/users/8a4e5ca94a4d4e3fa0026821ce93f562/roles/609a4bf154ff4dcabb408b3789d28983` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:19.198631 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319688d-6b48-48a0-bd14-defafcd5f2b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a1236f1bb0f4a499c0206128e5c97f9, user_id=8a4e5ca94a4d4e3fa0026821ce93f562, role_id=609a4bf154ff4dcabb408b3789d28983)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:19.199877 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3319688d-6b48-48a0-bd14-defafcd5f2b9 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:19.256329 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 78/159] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Oct 17 03:32:19 2022] PUT /identity/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9/users/8a4e5ca94a4d4e3fa0026821ce93f562/roles/609a4bf154ff4dcabb408b3789d28983 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:32:21.049002 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df510102-9c36-442c-b245-24bf2d909956 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:21.049559 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df510102-9c36-442c-b245-24bf2d909956 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:21.049559 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df510102-9c36-442c-b245-24bf2d909956 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:21.050956 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 82/160] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:21.057059 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0a431-186b-4962-9d5c-5f3e485a762e None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:21.057619 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0a431-186b-4962-9d5c-5f3e485a762e None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:21.057849 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0a431-186b-4962-9d5c-5f3e485a762e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:21.079960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-8ee0a431-186b-4962-9d5c-5f3e485a762e None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:21.081206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-8ee0a431-186b-4962-9d5c-5f3e485a762e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:21.164836 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 79/161] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:21.187702 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8477f64-f753-44a9-8f2f-a1529c6e0908 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:21.188032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8477f64-f753-44a9-8f2f-a1529c6e0908 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:21.188282 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8477f64-f753-44a9-8f2f-a1529c6e0908 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:21.189634 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 83/162] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:21.195100 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a16f7d-4f55-4092-9ee0-e14f34af64a9 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:21.195464 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a16f7d-4f55-4092-9ee0-e14f34af64a9 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:21.195842 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a16f7d-4f55-4092-9ee0-e14f34af64a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:21.214736 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-41a16f7d-4f55-4092-9ee0-e14f34af64a9 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:21.216258 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-41a16f7d-4f55-4092-9ee0-e14f34af64a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:21.228124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:21.240797 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b419cc-56d2-4b6c-a8c8-01ba6b7928b2 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:21.241753 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b419cc-56d2-4b6c-a8c8-01ba6b7928b2 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:21.241753 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b419cc-56d2-4b6c-a8c8-01ba6b7928b2 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:21.242716 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 84/163] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:32:21.248429 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdaeae9-c8dd-4046-8b37-40c1c85b98af None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:21.248548 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdaeae9-c8dd-4046-8b37-40c1c85b98af None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:21.248773 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdaeae9-c8dd-4046-8b37-40c1c85b98af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:21.269932 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-cfdaeae9-c8dd-4046-8b37-40c1c85b98af None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:21.271158 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-cfdaeae9-c8dd-4046-8b37-40c1c85b98af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:21.296713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 80/164] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:21.350245 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 85/165] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 03:32:21.395877 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8752e5-7e52-482d-b784-d8d12ea1184b None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:21.396404 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8752e5-7e52-482d-b784-d8d12ea1184b None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:21.396863 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8752e5-7e52-482d-b784-d8d12ea1184b None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:21.398849 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 81/166] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:21.404464 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131be0dd-20a0-40a3-9f2f-2c145ae34e10 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:21.404753 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131be0dd-20a0-40a3-9f2f-2c145ae34e10 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:21.405029 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131be0dd-20a0-40a3-9f2f-2c145ae34e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:21.423669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-131be0dd-20a0-40a3-9f2f-2c145ae34e10 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:21.424876 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-131be0dd-20a0-40a3-9f2f-2c145ae34e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:21.435686 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:21.447187 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8752e5-7e52-482d-b784-d8d12ea1184b None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:21.501142 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 86/167] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:21.552813 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8752e5-7e52-482d-b784-d8d12ea1184b None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:21.574685 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131be0dd-20a0-40a3-9f2f-2c145ae34e10 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:21.590244 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dabe2a-df07-478f-ba80-5e2a4e8145cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:21.591945 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dabe2a-df07-478f-ba80-5e2a4e8145cf None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:21.592230 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dabe2a-df07-478f-ba80-5e2a4e8145cf None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:21.592506 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dabe2a-df07-478f-ba80-5e2a4e8145cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:21.607269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dabe2a-df07-478f-ba80-5e2a4e8145cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:21.613573 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-32dabe2a-df07-478f-ba80-5e2a4e8145cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 03:32:21.613573 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:21.613573 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:21.613573 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:21.613573 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:21.613573 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:21.613573 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:21.613573 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:21.613573 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:21.613573 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:21.613573 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:21.613573 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:21.613573 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:21.613573 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:21.613573 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:21.613573 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:21.614463 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:21.614463 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:21.614463 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:21.614463 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:21.614463 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:21.614463 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:21.614463 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:21.614463 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:21.614463 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:21.614463 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:21.614463 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:21.614463 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:21.614463 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:21.614463 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:21.614463 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:21.614463 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:21.615249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:21.615249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:21.615249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:21.615249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:21.615249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:21.615249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:21.615249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:21.615249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:21.615249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:21.615249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:21.615249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 03:32:21.615249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:21.615249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 82/168] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:32:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:21.620671 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dabe2a-df07-478f-ba80-5e2a4e8145cf None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:21.632333 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:21.681845 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131be0dd-20a0-40a3-9f2f-2c145ae34e10 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:21.800448 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e518ed3-cee2-424e-aff0-06caad7d04ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:21.801076 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e518ed3-cee2-424e-aff0-06caad7d04ed None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:21.801349 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e518ed3-cee2-424e-aff0-06caad7d04ed None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:21.801616 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e518ed3-cee2-424e-aff0-06caad7d04ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:21.802759 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e518ed3-cee2-424e-aff0-06caad7d04ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:21.804277 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e518ed3-cee2-424e-aff0-06caad7d04ed None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:21.809123 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 87/169] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:32:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:21.809123 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e518ed3-cee2-424e-aff0-06caad7d04ed None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:21.823274 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:21.833639 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dabe2a-df07-478f-ba80-5e2a4e8145cf None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:21.868771 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a81bf-daf3-4779-bc4d-71038b200454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:21.869308 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a81bf-daf3-4779-bc4d-71038b200454 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:21.869592 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a81bf-daf3-4779-bc4d-71038b200454 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:21.869866 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a81bf-daf3-4779-bc4d-71038b200454 None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:21.870536 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a81bf-daf3-4779-bc4d-71038b200454 None admin] RBAC: Adding query filter params (name=member) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:21.870839 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a81bf-daf3-4779-bc4d-71038b200454 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:21.872352 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a81bf-daf3-4779-bc4d-71038b200454 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:21.880581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 83/170] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Oct 17 03:32:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:21.885229 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a81bf-daf3-4779-bc4d-71038b200454 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:21.909893 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e518ed3-cee2-424e-aff0-06caad7d04ed None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:21.942304 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535f88f-4ed2-423e-92dd-524903f5e6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:21.942495 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:21.943214 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535f88f-4ed2-423e-92dd-524903f5e6f0 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:21.943506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535f88f-4ed2-423e-92dd-524903f5e6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:21.943819 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535f88f-4ed2-423e-92dd-524903f5e6f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:21.954272 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535f88f-4ed2-423e-92dd-524903f5e6f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:21.959192 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-d535f88f-4ed2-423e-92dd-524903f5e6f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:32:21.959192 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:21.959192 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:21.959192 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:21.959192 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:21.959192 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:21.959192 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:21.959192 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:21.959192 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:21.959192 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:21.959192 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:21.959192 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:21.959192 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:21.959192 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:21.959192 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:21.959192 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:21.960234 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:21.960234 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:21.960234 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:21.960234 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:21.960234 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:21.960234 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:21.960234 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:21.960234 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:21.960234 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:21.960234 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:21.960234 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:21.960234 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:21.960234 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:21.960234 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:21.960234 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:21.960234 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:21.960997 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:21.960997 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:21.960997 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:21.960997 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:21.960997 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:21.960997 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:21.960997 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:21.960997 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:21.960997 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:21.960997 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:21.960997 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:32:21.960997 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:21.960997 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 88/171] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Oct 17 03:32:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:21.966709 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535f88f-4ed2-423e-92dd-524903f5e6f0 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:21.980343 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a81bf-daf3-4779-bc4d-71038b200454 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:22.013474 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd11f9-6461-412f-b866-ad4018d790d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:22.014104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd11f9-6461-412f-b866-ad4018d790d3 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:22.014388 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd11f9-6461-412f-b866-ad4018d790d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:22.014662 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd11f9-6461-412f-b866-ad4018d790d3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:22.024530 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd11f9-6461-412f-b866-ad4018d790d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:22.030328 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-aadd11f9-6461-412f-b866-ad4018d790d3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 03:32:22.030328 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:22.030328 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:22.030328 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:22.030328 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:22.030328 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:22.030328 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:22.030328 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:22.030328 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:22.030328 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:22.030328 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:22.030328 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:22.030328 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:22.030328 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:22.030328 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:22.030328 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:22.031231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:22.031231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:22.031231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:22.031231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:22.031231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:22.031231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:22.031231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:22.031231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:22.031231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:22.031231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:22.031231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:22.031231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:22.031231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:22.031231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:22.031231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:22.031231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:22.032043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:22.032043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:22.032043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:22.032043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:22.032043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:22.032043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:22.032043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:22.032043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:22.032043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:22.032043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:22.032043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 03:32:22.032043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:22.032043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 84/172] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Oct 17 03:32:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:22.032689 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd11f9-6461-412f-b866-ad4018d790d3 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:22.042823 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:22.066921 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535f88f-4ed2-423e-92dd-524903f5e6f0 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:22.101400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661bcb9-3275-4034-9687-e76737d2b409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:22.102013 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661bcb9-3275-4034-9687-e76737d2b409 None admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:22.102296 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661bcb9-3275-4034-9687-e76737d2b409 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:22.102571 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661bcb9-3275-4034-9687-e76737d2b409 None admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:22.103226 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0661bcb9-3275-4034-9687-e76737d2b409 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:22.104694 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0661bcb9-3275-4034-9687-e76737d2b409 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:22.130598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd11f9-6461-412f-b866-ad4018d790d3 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:22.154402 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:22.164667 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c66772-8bd4-46a8-86dd-d9afb567dc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:22.165287 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c66772-8bd4-46a8-86dd-d9afb567dc78 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:22.165568 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c66772-8bd4-46a8-86dd-d9afb567dc78 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:22.165838 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c66772-8bd4-46a8-86dd-d9afb567dc78 None admin] PATH_INFO: `/v3/users/e5f413fcbcae4e8280104cd21b345199` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:22.168454 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 89/173] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Oct 17 03:32:21 2022] POST /identity/v3/users => generated 297 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:32:22.169309 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661bcb9-3275-4034-9687-e76737d2b409 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:22.180205 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c66772-8bd4-46a8-86dd-d9afb567dc78 None admin] RBAC: Authorizing `identity:get_user(user_id=e5f413fcbcae4e8280104cd21b345199)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:22.181733 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c66772-8bd4-46a8-86dd-d9afb567dc78 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:22.183481 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 85/174] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:32:22 2022] GET /identity/v3/users/e5f413fcbcae4e8280104cd21b345199 => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:22.184332 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c66772-8bd4-46a8-86dd-d9afb567dc78 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:22.295555 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c66772-8bd4-46a8-86dd-d9afb567dc78 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:22.298144 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661bcb9-3275-4034-9687-e76737d2b409 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:22.331812 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:22.333936 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbdd35-3480-4857-832d-c0a1fe1063b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:22.334554 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbdd35-3480-4857-832d-c0a1fe1063b8 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:22.334837 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbdd35-3480-4857-832d-c0a1fe1063b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:22.335104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdbdd35-3480-4857-832d-c0a1fe1063b8 None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:22.335846 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdbdd35-3480-4857-832d-c0a1fe1063b8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:22.336659 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdbdd35-3480-4857-832d-c0a1fe1063b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:22.341107 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdbdd35-3480-4857-832d-c0a1fe1063b8 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:22.341217 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c7054-fce1-4ce2-b667-2d1ca41bc3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:22.341217 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c7054-fce1-4ce2-b667-2d1ca41bc3c5 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:22.341217 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c7054-fce1-4ce2-b667-2d1ca41bc3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:22.341217 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c7054-fce1-4ce2-b667-2d1ca41bc3c5 None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:22.341491 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c7054-fce1-4ce2-b667-2d1ca41bc3c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:22.341650 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c7054-fce1-4ce2-b667-2d1ca41bc3c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:22.343003 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c7054-fce1-4ce2-b667-2d1ca41bc3c5 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:22.346462 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 86/175] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:32:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:22.347359 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbdd35-3480-4857-832d-c0a1fe1063b8 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:22.362131 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 90/176] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:32:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:22.363290 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c7054-fce1-4ce2-b667-2d1ca41bc3c5 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:22.403987 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:22.464660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdbdd35-3480-4857-832d-c0a1fe1063b8 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:22.487254 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c7054-fce1-4ce2-b667-2d1ca41bc3c5 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:22.504447 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528138b-9f86-4b68-b485-51c618185c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:22.505086 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528138b-9f86-4b68-b485-51c618185c10 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:22.505364 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528138b-9f86-4b68-b485-51c618185c10 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:22.505636 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c528138b-9f86-4b68-b485-51c618185c10 None admin] PATH_INFO: `/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:22.507544 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528138b-9f86-4b68-b485-51c618185c10 None admin] RBAC: Authorizing `identity:get_project(project_id=9a1236f1bb0f4a499c0206128e5c97f9)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:22.509027 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c528138b-9f86-4b68-b485-51c618185c10 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:22.510838 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 87/177] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:32:22 2022] GET /identity/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:22.511608 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528138b-9f86-4b68-b485-51c618185c10 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:22.525916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07784a1b-6e19-47d9-84b7-6b772f663392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:22.526551 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07784a1b-6e19-47d9-84b7-6b772f663392 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:22.526827 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07784a1b-6e19-47d9-84b7-6b772f663392 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:22.527091 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07784a1b-6e19-47d9-84b7-6b772f663392 None admin] PATH_INFO: `/v3/users/8a4e5ca94a4d4e3fa0026821ce93f562` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:22.528314 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07784a1b-6e19-47d9-84b7-6b772f663392 None admin] RBAC: Authorizing `identity:get_user(user_id=8a4e5ca94a4d4e3fa0026821ce93f562)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:22.529730 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07784a1b-6e19-47d9-84b7-6b772f663392 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:22.531446 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 91/178] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:32:22 2022] GET /identity/v3/users/8a4e5ca94a4d4e3fa0026821ce93f562 => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:22.532602 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07784a1b-6e19-47d9-84b7-6b772f663392 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:22.607414 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:22.627937 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c528138b-9f86-4b68-b485-51c618185c10 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:22.644232 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:22.655795 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07784a1b-6e19-47d9-84b7-6b772f663392 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:22.668502 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b16ab0-3048-446d-a6f7-2ab9b6d0067e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:22.669779 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b16ab0-3048-446d-a6f7-2ab9b6d0067e None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:22.670101 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b16ab0-3048-446d-a6f7-2ab9b6d0067e None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:22.670434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b16ab0-3048-446d-a6f7-2ab9b6d0067e None admin] PATH_INFO: `/v3/users/8a4e5ca94a4d4e3fa0026821ce93f562` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:22.671849 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b16ab0-3048-446d-a6f7-2ab9b6d0067e None admin] RBAC: Authorizing `identity:get_user(user_id=8a4e5ca94a4d4e3fa0026821ce93f562)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:22.673256 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b16ab0-3048-446d-a6f7-2ab9b6d0067e None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:22.675066 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 88/179] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:32:22 2022] GET /identity/v3/users/8a4e5ca94a4d4e3fa0026821ce93f562 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:22.675650 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b16ab0-3048-446d-a6f7-2ab9b6d0067e None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:22.700321 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7046cd-0eab-4ec8-b439-26eb28289e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:22.701098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7046cd-0eab-4ec8-b439-26eb28289e5e None admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:22.701226 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7046cd-0eab-4ec8-b439-26eb28289e5e None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:22.701480 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7046cd-0eab-4ec8-b439-26eb28289e5e None admin] PATH_INFO: `/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9/users/e5f413fcbcae4e8280104cd21b345199/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:22.706189 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7046cd-0eab-4ec8-b439-26eb28289e5e None admin] RBAC: Authorizing `identity:create_grant(project_id=9a1236f1bb0f4a499c0206128e5c97f9, user_id=e5f413fcbcae4e8280104cd21b345199, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:22.707640 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7046cd-0eab-4ec8-b439-26eb28289e5e None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:22.777718 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 92/180] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Oct 17 03:32:22 2022] PUT /identity/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9/users/e5f413fcbcae4e8280104cd21b345199/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:32:22.779065 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7046cd-0eab-4ec8-b439-26eb28289e5e None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:22.794272 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b16ab0-3048-446d-a6f7-2ab9b6d0067e None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:22.872367 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745d7f0-4b9f-4e50-a3f9-3102b0c39939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:22.873008 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745d7f0-4b9f-4e50-a3f9-3102b0c39939 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:22.873290 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745d7f0-4b9f-4e50-a3f9-3102b0c39939 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:22.873563 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745d7f0-4b9f-4e50-a3f9-3102b0c39939 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:22.949039 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:22.964748 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c745d7f0-4b9f-4e50-a3f9-3102b0c39939 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:22.971136 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c745d7f0-4b9f-4e50-a3f9-3102b0c39939 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:23.010035 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7046cd-0eab-4ec8-b439-26eb28289e5e None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:23.042294 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:23.060462 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa0db0-cd32-4240-ae35-4a032679fd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:23.061086 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa0db0-cd32-4240-ae35-4a032679fd80 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:23.061368 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa0db0-cd32-4240-ae35-4a032679fd80 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:23.061641 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa0db0-cd32-4240-ae35-4a032679fd80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:23.108815 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 89/181] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:32:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 03:32:23.114356 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745d7f0-4b9f-4e50-a3f9-3102b0c39939 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:23.137808 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efa0db0-cd32-4240-ae35-4a032679fd80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:23.142748 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efa0db0-cd32-4240-ae35-4a032679fd80 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:23.200669 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:23.222117 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745d7f0-4b9f-4e50-a3f9-3102b0c39939 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:23.243690 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 93/182] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:32:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 03:32:23.249441 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa0db0-cd32-4240-ae35-4a032679fd80 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:23.259927 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac512d1-b657-476f-882b-b10dca04ff50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:23.260960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac512d1-b657-476f-882b-b10dca04ff50 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:23.261435 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac512d1-b657-476f-882b-b10dca04ff50 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:23.261929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac512d1-b657-476f-882b-b10dca04ff50 None admin] PATH_INFO: `/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:23.263847 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac512d1-b657-476f-882b-b10dca04ff50 None admin] RBAC: Authorizing `identity:get_project(project_id=9a1236f1bb0f4a499c0206128e5c97f9)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:23.265523 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac512d1-b657-476f-882b-b10dca04ff50 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:23.267453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 90/183] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:32:23 2022] GET /identity/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:23.273349 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac512d1-b657-476f-882b-b10dca04ff50 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:23.284152 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:23.359906 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa0db0-cd32-4240-ae35-4a032679fd80 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:23.433831 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac512d1-b657-476f-882b-b10dca04ff50 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:23.436468 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5100e0-605a-445b-9e9d-5e14b95c9346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:23.437119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5100e0-605a-445b-9e9d-5e14b95c9346 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:23.437396 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5100e0-605a-445b-9e9d-5e14b95c9346 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:23.437668 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5100e0-605a-445b-9e9d-5e14b95c9346 None admin] PATH_INFO: `/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:23.439589 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5100e0-605a-445b-9e9d-5e14b95c9346 None admin] RBAC: Authorizing `identity:get_project(project_id=9a1236f1bb0f4a499c0206128e5c97f9)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:23.441082 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5100e0-605a-445b-9e9d-5e14b95c9346 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:23.453787 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 94/184] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:32:23 2022] GET /identity/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:23.454590 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5100e0-605a-445b-9e9d-5e14b95c9346 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:23.503450 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e3ba0-54ef-4887-b44b-9c13b3906449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:23.504095 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e3ba0-54ef-4887-b44b-9c13b3906449 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:23.504385 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e3ba0-54ef-4887-b44b-9c13b3906449 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:23.504662 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e3ba0-54ef-4887-b44b-9c13b3906449 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:23.505401 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e3ba0-54ef-4887-b44b-9c13b3906449 None admin] RBAC: Adding query filter params (role.id=609a4bf154ff4dcabb408b3789d28983, scope.project.id=9a1236f1bb0f4a499c0206128e5c97f9, user.id=8a4e5ca94a4d4e3fa0026821ce93f562) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:23.505712 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e3ba0-54ef-4887-b44b-9c13b3906449 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:23.507164 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e3ba0-54ef-4887-b44b-9c13b3906449 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:23.528025 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 91/185] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Oct 17 03:32:23 2022] GET /identity/v3/role_assignments?user.id=8a4e5ca94a4d4e3fa0026821ce93f562&scope.project.id=9a1236f1bb0f4a499c0206128e5c97f9&role.id=609a4bf154ff4dcabb408b3789d28983 => generated 608 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:23.614721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:23.632731 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5100e0-605a-445b-9e9d-5e14b95c9346 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:23.692326 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e969ef1-a158-44c8-8040-a77dc2b5e448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:23.692968 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e969ef1-a158-44c8-8040-a77dc2b5e448 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:23.693250 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e969ef1-a158-44c8-8040-a77dc2b5e448 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:23.693523 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e969ef1-a158-44c8-8040-a77dc2b5e448 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:23.694284 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e969ef1-a158-44c8-8040-a77dc2b5e448 None admin] RBAC: Adding query filter params (role.id=611120afb3a544d0b5161da1b769a2f8, scope.project.id=9a1236f1bb0f4a499c0206128e5c97f9, user.id=8a4e5ca94a4d4e3fa0026821ce93f562) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:23.694596 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e969ef1-a158-44c8-8040-a77dc2b5e448 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:23.696078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e969ef1-a158-44c8-8040-a77dc2b5e448 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:23.710621 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 95/186] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Oct 17 03:32:23 2022] GET /identity/v3/role_assignments?user.id=8a4e5ca94a4d4e3fa0026821ce93f562&scope.project.id=9a1236f1bb0f4a499c0206128e5c97f9&role.id=611120afb3a544d0b5161da1b769a2f8 => generated 608 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:24.608698 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb94b9b-754e-47aa-a451-bb10531d7984 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:24.609230 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb94b9b-754e-47aa-a451-bb10531d7984 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:24.609230 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb94b9b-754e-47aa-a451-bb10531d7984 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:24.610776 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 92/187] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:24.616604 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11411b29-f51f-4388-a9b1-1f88bbdcceb3 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:24.617128 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11411b29-f51f-4388-a9b1-1f88bbdcceb3 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:24.617211 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11411b29-f51f-4388-a9b1-1f88bbdcceb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:24.635650 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-11411b29-f51f-4388-a9b1-1f88bbdcceb3 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:24.636882 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-11411b29-f51f-4388-a9b1-1f88bbdcceb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:24.711081 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 96/188] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:24.711261 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0bb90-97a8-492f-9aad-eada923fe344 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:24.711421 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0bb90-97a8-492f-9aad-eada923fe344 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:24.711695 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0bb90-97a8-492f-9aad-eada923fe344 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:24.713313 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 93/189] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:32:24.718543 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfcd51-8253-4285-a99a-6d429281b2c8 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:24.718826 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfcd51-8253-4285-a99a-6d429281b2c8 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:24.719101 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfcd51-8253-4285-a99a-6d429281b2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:24.737829 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-3ecfcd51-8253-4285-a99a-6d429281b2c8 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:24.739003 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-3ecfcd51-8253-4285-a99a-6d429281b2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:24.746498 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:24.806703 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 94/190] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 03:32:24.979147 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11411b29-f51f-4388-a9b1-1f88bbdcceb3 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:25.058394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:25.074020 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecfcd51-8253-4285-a99a-6d429281b2c8 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:25.077104 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11411b29-f51f-4388-a9b1-1f88bbdcceb3 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:25.115842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbc4f5-3bf4-47c6-8db5-02cce7bdaeac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:25.116483 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbc4f5-3bf4-47c6-8db5-02cce7bdaeac None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:25.116765 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbc4f5-3bf4-47c6-8db5-02cce7bdaeac None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:25.117047 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbc4f5-3bf4-47c6-8db5-02cce7bdaeac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:25.128490 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecbc4f5-3bf4-47c6-8db5-02cce7bdaeac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:25.133388 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-2ecbc4f5-3bf4-47c6-8db5-02cce7bdaeac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 03:32:25.133388 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:25.133388 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:25.133388 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:25.133388 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:25.133388 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:25.133388 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:25.133388 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:25.133388 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:25.133388 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:25.133388 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:25.133388 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:25.133388 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:25.133388 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:25.133388 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:25.133388 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:25.134313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:25.134313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:25.134313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:25.134313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:25.134313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:25.134313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:25.134313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:25.134313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:25.134313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:25.134313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:25.134313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:25.134313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:25.134313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:25.134313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:25.134313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:25.134313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:25.135101 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:25.135101 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:25.135101 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:25.135101 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:25.135101 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:25.135101 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:25.135101 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:25.135101 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:25.135101 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:25.135101 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:25.135101 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 03:32:25.135101 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:25.135101 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 97/191] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:32:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:25.140184 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbc4f5-3bf4-47c6-8db5-02cce7bdaeac None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:25.180898 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecfcd51-8253-4285-a99a-6d429281b2c8 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:25.218828 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d2df3-fefc-44e0-baf2-804893cb867c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:25.219477 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d2df3-fefc-44e0-baf2-804893cb867c None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:25.219785 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d2df3-fefc-44e0-baf2-804893cb867c None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:25.220054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d2df3-fefc-44e0-baf2-804893cb867c None admin] PATH_INFO: `/v3/domains/default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:25.221707 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960d2df3-fefc-44e0-baf2-804893cb867c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:25.223575 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960d2df3-fefc-44e0-baf2-804893cb867c None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:25.225757 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 95/192] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:32:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:25.230878 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d2df3-fefc-44e0-baf2-804893cb867c None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:25.244080 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:25.248920 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbc4f5-3bf4-47c6-8db5-02cce7bdaeac None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:25.285649 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adc2c1d-59a1-4314-9850-a123c6c597b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:25.286527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc2c1d-59a1-4314-9850-a123c6c597b9 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:25.286804 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc2c1d-59a1-4314-9850-a123c6c597b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:25.287076 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc2c1d-59a1-4314-9850-a123c6c597b9 None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:25.287756 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adc2c1d-59a1-4314-9850-a123c6c597b9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:25.288063 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adc2c1d-59a1-4314-9850-a123c6c597b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:25.289417 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adc2c1d-59a1-4314-9850-a123c6c597b9 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:25.298321 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 98/193] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Oct 17 03:32:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:25.303754 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adc2c1d-59a1-4314-9850-a123c6c597b9 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:25.307268 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:25.338149 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d2df3-fefc-44e0-baf2-804893cb867c None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:25.371594 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17620c51-b3e9-468e-bcb7-17f69b838cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:25.372227 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17620c51-b3e9-468e-bcb7-17f69b838cbe None admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:25.372503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17620c51-b3e9-468e-bcb7-17f69b838cbe None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:25.372774 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17620c51-b3e9-468e-bcb7-17f69b838cbe None admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:25.373420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17620c51-b3e9-468e-bcb7-17f69b838cbe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:25.374841 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17620c51-b3e9-468e-bcb7-17f69b838cbe None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:25.402706 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adc2c1d-59a1-4314-9850-a123c6c597b9 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:25.436568 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf5e85-915b-4710-8ad9-5f204e10a1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:25.437193 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf5e85-915b-4710-8ad9-5f204e10a1d9 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:25.437574 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf5e85-915b-4710-8ad9-5f204e10a1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:25.437860 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf5e85-915b-4710-8ad9-5f204e10a1d9 None admin] PATH_INFO: `/v3/users/8235738370424e138c69ad6bd47b3119` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:25.447513 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 96/194] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Oct 17 03:32:25 2022] POST /identity/v3/users => generated 293 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:32:25.448767 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa314e-4825-412a-9dc1-6afa83805059 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:25.449048 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa314e-4825-412a-9dc1-6afa83805059 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:25.449316 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffa314e-4825-412a-9dc1-6afa83805059 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:25.450539 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 97/195] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:25.456426 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509fb64-7b53-419e-8691-6c20bcf68e4d None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:25.456749 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509fb64-7b53-419e-8691-6c20bcf68e4d None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:25.456981 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509fb64-7b53-419e-8691-6c20bcf68e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:25.462063 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf5e85-915b-4710-8ad9-5f204e10a1d9 None admin] RBAC: Authorizing `identity:get_user(user_id=8235738370424e138c69ad6bd47b3119)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:25.463874 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf5e85-915b-4710-8ad9-5f204e10a1d9 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:25.472555 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 99/196] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:32:25 2022] GET /identity/v3/users/8235738370424e138c69ad6bd47b3119 => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:25.473760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf5e85-915b-4710-8ad9-5f204e10a1d9 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:25.476994 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-4509fb64-7b53-419e-8691-6c20bcf68e4d None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:25.478216 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-4509fb64-7b53-419e-8691-6c20bcf68e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:25.543362 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:25.548202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:25.556083 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 98/197] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:25.583884 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf5e85-915b-4710-8ad9-5f204e10a1d9 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:25.623585 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b51339-ec4b-4c3a-8a54-b7860a03abde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:25.624394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b51339-ec4b-4c3a-8a54-b7860a03abde None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:25.625863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b51339-ec4b-4c3a-8a54-b7860a03abde None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:25.625863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b51339-ec4b-4c3a-8a54-b7860a03abde None admin] PATH_INFO: `/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:25.627119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b51339-ec4b-4c3a-8a54-b7860a03abde None admin] RBAC: Authorizing `identity:get_project(project_id=9a1236f1bb0f4a499c0206128e5c97f9)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:25.628721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b51339-ec4b-4c3a-8a54-b7860a03abde None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:25.630173 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 100/198] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:32:25 2022] GET /identity/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:25.635599 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509fb64-7b53-419e-8691-6c20bcf68e4d None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:25.764409 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509fb64-7b53-419e-8691-6c20bcf68e4d None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:25.802789 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496abc3-4f97-424b-9385-ce93af7d8bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:25.803376 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496abc3-4f97-424b-9385-ce93af7d8bb8 None admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:25.803656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496abc3-4f97-424b-9385-ce93af7d8bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:25.803960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496abc3-4f97-424b-9385-ce93af7d8bb8 None admin] PATH_INFO: `/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9/users/8235738370424e138c69ad6bd47b3119/roles/010a82aa9f3b4a23817fe234196cf259` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:25.805910 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496abc3-4f97-424b-9385-ce93af7d8bb8 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a1236f1bb0f4a499c0206128e5c97f9, user_id=8235738370424e138c69ad6bd47b3119, role_id=010a82aa9f3b4a23817fe234196cf259)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:25.807415 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496abc3-4f97-424b-9385-ce93af7d8bb8 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:25.825944 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b51339-ec4b-4c3a-8a54-b7860a03abde None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:25.877721 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 99/199] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Oct 17 03:32:25 2022] PUT /identity/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9/users/8235738370424e138c69ad6bd47b3119/roles/010a82aa9f3b4a23817fe234196cf259 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:32:25.938105 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b51339-ec4b-4c3a-8a54-b7860a03abde None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:25.972149 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:25.977487 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03c19e-93b9-478a-8ae1-f04499fef5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:25.978144 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03c19e-93b9-478a-8ae1-f04499fef5f2 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:25.978425 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03c19e-93b9-478a-8ae1-f04499fef5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:25.978699 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b03c19e-93b9-478a-8ae1-f04499fef5f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:25.991698 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b03c19e-93b9-478a-8ae1-f04499fef5f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:25.996864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-6b03c19e-93b9-478a-8ae1-f04499fef5f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 03:32:25.996864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:25.996864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:25.996864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:25.996864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:25.996864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:25.996864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:25.996864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:25.996864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:25.996864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:25.996864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:25.996864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:25.996864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:25.996864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:25.996864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:25.996864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:25.997753 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:25.997753 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:25.997753 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:25.997753 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:25.997753 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:25.997753 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:25.997753 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:25.997753 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:25.997753 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:25.997753 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:25.997753 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:25.997753 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:25.997753 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:25.997753 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:25.997753 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:25.997753 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:25.998531 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:25.998531 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:25.998531 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:25.998531 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:25.998531 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:25.998531 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:25.998531 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:25.998531 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:25.998531 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:25.998531 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:25.998531 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 03:32:25.998531 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:25.998531 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 101/200] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:32:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:32:26.004851 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496abc3-4f97-424b-9385-ce93af7d8bb8 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:26.068913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:26.118960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496abc3-4f97-424b-9385-ce93af7d8bb8 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:26.154598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d31979-ecf5-48a9-b826-ca65cc944089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:26.155216 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d31979-ecf5-48a9-b826-ca65cc944089 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:26.155494 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d31979-ecf5-48a9-b826-ca65cc944089 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:26.155789 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d31979-ecf5-48a9-b826-ca65cc944089 None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:26.156523 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d31979-ecf5-48a9-b826-ca65cc944089 None admin] RBAC: Adding query filter params (name=member) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:26.156827 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d31979-ecf5-48a9-b826-ca65cc944089 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:26.158185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d31979-ecf5-48a9-b826-ca65cc944089 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:26.166756 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 100/201] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Oct 17 03:32:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:26.173207 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03c19e-93b9-478a-8ae1-f04499fef5f2 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:26.280814 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b03c19e-93b9-478a-8ae1-f04499fef5f2 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:26.315491 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d618e-8efd-43f7-b785-b5c0b4a04936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:26.316164 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d618e-8efd-43f7-b785-b5c0b4a04936 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:26.316446 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d618e-8efd-43f7-b785-b5c0b4a04936 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:26.316719 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d618e-8efd-43f7-b785-b5c0b4a04936 None admin] PATH_INFO: `/v3/users/e5f413fcbcae4e8280104cd21b345199` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:26.317872 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d618e-8efd-43f7-b785-b5c0b4a04936 None admin] RBAC: Authorizing `identity:get_user(user_id=e5f413fcbcae4e8280104cd21b345199)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:26.319362 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d618e-8efd-43f7-b785-b5c0b4a04936 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:26.321110 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 102/202] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:32:26 2022] GET /identity/v3/users/e5f413fcbcae4e8280104cd21b345199 => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:26.327086 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d31979-ecf5-48a9-b826-ca65cc944089 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:26.432035 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d31979-ecf5-48a9-b826-ca65cc944089 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:26.449670 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:26.467679 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc48e17-09ec-40c6-a3cf-1642871fa0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:26.468324 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc48e17-09ec-40c6-a3cf-1642871fa0e4 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:26.468604 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc48e17-09ec-40c6-a3cf-1642871fa0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:26.468978 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc48e17-09ec-40c6-a3cf-1642871fa0e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:26.541463 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc48e17-09ec-40c6-a3cf-1642871fa0e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:26.542935 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc48e17-09ec-40c6-a3cf-1642871fa0e4 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:26.646468 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 101/203] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:32:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 17 03:32:26.651869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d618e-8efd-43f7-b785-b5c0b4a04936 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:26.751566 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:26.794041 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d618e-8efd-43f7-b785-b5c0b4a04936 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:26.829108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda9f25-6019-401e-bb4f-fd1c6370984c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:26.829740 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda9f25-6019-401e-bb4f-fd1c6370984c None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:26.830063 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda9f25-6019-401e-bb4f-fd1c6370984c None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:26.830338 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda9f25-6019-401e-bb4f-fd1c6370984c None admin] PATH_INFO: `/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:26.831452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda9f25-6019-401e-bb4f-fd1c6370984c None admin] RBAC: Authorizing `identity:get_project(project_id=9a1236f1bb0f4a499c0206128e5c97f9)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:26.832924 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fda9f25-6019-401e-bb4f-fd1c6370984c None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:26.834922 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 103/204] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:32:26 2022] GET /identity/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:26.840917 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc48e17-09ec-40c6-a3cf-1642871fa0e4 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:26.861178 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:26.945690 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc48e17-09ec-40c6-a3cf-1642871fa0e4 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:26.990611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2c2d7-8830-488f-b7bc-ecedd22b2d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:26.991224 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2c2d7-8830-488f-b7bc-ecedd22b2d69 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:26.991506 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2c2d7-8830-488f-b7bc-ecedd22b2d69 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:26.991807 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2c2d7-8830-488f-b7bc-ecedd22b2d69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:26.992636 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c2c2d7-8830-488f-b7bc-ecedd22b2d69 None admin] RBAC: Adding query filter params (role.id=84a147decc7b417783086c1effe0fe20, scope.project.id=9a1236f1bb0f4a499c0206128e5c97f9, user.id=e5f413fcbcae4e8280104cd21b345199) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:26.992943 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c2c2d7-8830-488f-b7bc-ecedd22b2d69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:26.994444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c2c2d7-8830-488f-b7bc-ecedd22b2d69 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:27.001290 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 102/205] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Oct 17 03:32:26 2022] GET /identity/v3/role_assignments?user.id=e5f413fcbcae4e8280104cd21b345199&scope.project.id=9a1236f1bb0f4a499c0206128e5c97f9&role.id=84a147decc7b417783086c1effe0fe20 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:27.818239 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b8914-ce7e-4d48-a96e-d4960b70a8ee None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:27.818780 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b8914-ce7e-4d48-a96e-d4960b70a8ee None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:27.818780 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b8914-ce7e-4d48-a96e-d4960b70a8ee None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:27.823697 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 104/206] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:27.826083 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b188c-6ec0-41c1-a7bc-2759453a3d0b None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:27.826490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b188c-6ec0-41c1-a7bc-2759453a3d0b None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:27.826663 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b188c-6ec0-41c1-a7bc-2759453a3d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:27.844820 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-bb0b188c-6ec0-41c1-a7bc-2759453a3d0b None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:27.845978 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-bb0b188c-6ec0-41c1-a7bc-2759453a3d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:27.878387 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc20222-120e-4a83-b8dd-3653a41cac98 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:27.878673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc20222-120e-4a83-b8dd-3653a41cac98 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:27.878948 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc20222-120e-4a83-b8dd-3653a41cac98 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:27.880438 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 105/207] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:32:27.885198 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc71d5-0a2b-4b69-a0ec-e650c66d37d1 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:27.885482 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc71d5-0a2b-4b69-a0ec-e650c66d37d1 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:27.885757 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc71d5-0a2b-4b69-a0ec-e650c66d37d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:27.906399 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-77cc71d5-0a2b-4b69-a0ec-e650c66d37d1 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:27.907932 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-77cc71d5-0a2b-4b69-a0ec-e650c66d37d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:27.915428 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 103/208] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:27.976867 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 106/209] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 03:32:28.136564 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948728f-757e-4d95-b3e7-8d0e86e7aa6e None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:28.136906 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948728f-757e-4d95-b3e7-8d0e86e7aa6e None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:28.137146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948728f-757e-4d95-b3e7-8d0e86e7aa6e None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:28.139159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 104/210] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:28.144174 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328515fb-ba5a-4d42-8081-7deda6e8f925 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:28.144539 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328515fb-ba5a-4d42-8081-7deda6e8f925 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:28.144885 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328515fb-ba5a-4d42-8081-7deda6e8f925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:28.163109 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-328515fb-ba5a-4d42-8081-7deda6e8f925 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:28.164330 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-328515fb-ba5a-4d42-8081-7deda6e8f925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:28.183113 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948728f-757e-4d95-b3e7-8d0e86e7aa6e None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:28.198153 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:28.218782 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:28.239904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 107/211] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:28.239904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328515fb-ba5a-4d42-8081-7deda6e8f925 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:28.291778 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948728f-757e-4d95-b3e7-8d0e86e7aa6e None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:28.329087 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79663813-9407-4ac7-b0b2-f20770483507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:28.329886 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79663813-9407-4ac7-b0b2-f20770483507 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:28.330172 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79663813-9407-4ac7-b0b2-f20770483507 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:28.330733 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79663813-9407-4ac7-b0b2-f20770483507 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:28.344405 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79663813-9407-4ac7-b0b2-f20770483507 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:28.344734 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328515fb-ba5a-4d42-8081-7deda6e8f925 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:28.350259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-79663813-9407-4ac7-b0b2-f20770483507 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 03:32:28.350259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:28.350259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:28.350259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:28.350259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:28.350259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:28.350259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:28.350259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:28.350259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:28.350259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:28.350259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:28.350259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:28.350259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:28.350259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:28.350259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:28.350259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:28.351104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:28.351104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:28.351104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:28.351104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:28.351104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:28.351104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:28.351104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:28.351104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:28.351104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:28.351104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:28.351104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:28.351104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:28.351104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:28.351104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:28.351104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:28.351104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:28.351912 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:28.351912 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:28.351912 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:28.351912 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:28.351912 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:28.351912 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:28.351912 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:28.351912 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:28.351912 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:28.351912 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:28.351912 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 03:32:28.351912 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:28.351912 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 105/212] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:32:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:32:28.358582 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79663813-9407-4ac7-b0b2-f20770483507 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:28.383840 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95f4e9-5e92-4d69-903b-f83ecf938c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:28.384500 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95f4e9-5e92-4d69-903b-f83ecf938c94 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:28.384748 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95f4e9-5e92-4d69-903b-f83ecf938c94 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:28.385026 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95f4e9-5e92-4d69-903b-f83ecf938c94 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:28.386132 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa95f4e9-5e92-4d69-903b-f83ecf938c94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:28.388227 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa95f4e9-5e92-4d69-903b-f83ecf938c94 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:28.389965 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 108/213] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:32:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:28.395034 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95f4e9-5e92-4d69-903b-f83ecf938c94 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:28.440597 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:28.450635 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:28.470995 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79663813-9407-4ac7-b0b2-f20770483507 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:28.494558 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95f4e9-5e92-4d69-903b-f83ecf938c94 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:28.506431 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782f3cc-8ca7-4952-aa20-b5ea946993d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:28.507043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782f3cc-8ca7-4952-aa20-b5ea946993d3 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:28.507325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782f3cc-8ca7-4952-aa20-b5ea946993d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:28.507598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782f3cc-8ca7-4952-aa20-b5ea946993d3 None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:28.508292 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7782f3cc-8ca7-4952-aa20-b5ea946993d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:28.508598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7782f3cc-8ca7-4952-aa20-b5ea946993d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:28.509966 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7782f3cc-8ca7-4952-aa20-b5ea946993d3 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:28.518051 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 106/214] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Oct 17 03:32:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:28.518825 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782f3cc-8ca7-4952-aa20-b5ea946993d3 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:28.532377 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083c1bf-508a-4b40-af09-86d3984fdf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:28.532986 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083c1bf-508a-4b40-af09-86d3984fdf3f None admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:28.533263 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083c1bf-508a-4b40-af09-86d3984fdf3f None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:28.533534 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083c1bf-508a-4b40-af09-86d3984fdf3f None admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:28.534185 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083c1bf-508a-4b40-af09-86d3984fdf3f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:28.535598 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083c1bf-508a-4b40-af09-86d3984fdf3f None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:28.615361 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782f3cc-8ca7-4952-aa20-b5ea946993d3 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:28.636424 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 109/215] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Oct 17 03:32:28 2022] POST /identity/v3/users => generated 293 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:32:28.636424 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083c1bf-508a-4b40-af09-86d3984fdf3f None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:28.661504 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f76ce6-9696-40b4-a810-8ff05003251d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:28.662166 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f76ce6-9696-40b4-a810-8ff05003251d None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:28.662441 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f76ce6-9696-40b4-a810-8ff05003251d None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:28.662709 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f76ce6-9696-40b4-a810-8ff05003251d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:28.665127 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:28.676080 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f76ce6-9696-40b4-a810-8ff05003251d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:28.681260 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-e5f76ce6-9696-40b4-a810-8ff05003251d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 03:32:28.681260 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:28.681260 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:28.681260 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:28.681260 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:28.681260 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:28.681260 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:28.681260 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:28.681260 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:28.681260 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:28.681260 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:28.681260 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:28.681260 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:28.681260 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:28.681260 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:28.681260 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:28.682212 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:28.682212 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:28.682212 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:28.682212 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:28.682212 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:28.682212 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:28.682212 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:28.682212 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:28.682212 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:28.682212 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:28.682212 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:28.682212 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:28.682212 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:28.682212 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:28.682212 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:28.682212 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:28.683513 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:28.683513 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:28.683513 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:28.683513 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:28.683513 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:28.683513 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:28.683513 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:28.683513 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:28.683513 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:28.683513 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:28.683513 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 03:32:28.683513 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:28.683513 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 107/216] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:32:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:28.690342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f76ce6-9696-40b4-a810-8ff05003251d None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:28.768405 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083c1bf-508a-4b40-af09-86d3984fdf3f None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:28.781935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:28.808086 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f538ec-0c9f-40a7-9b48-f456df8f4c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:28.809616 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f538ec-0c9f-40a7-9b48-f456df8f4c1a None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:28.809908 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f538ec-0c9f-40a7-9b48-f456df8f4c1a None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:28.810183 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f538ec-0c9f-40a7-9b48-f456df8f4c1a None admin] PATH_INFO: `/v3/users/3dca63322132495abdb0b82bd99ef20a` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:28.811711 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f76ce6-9696-40b4-a810-8ff05003251d None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:28.827047 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f538ec-0c9f-40a7-9b48-f456df8f4c1a None admin] RBAC: Authorizing `identity:get_user(user_id=3dca63322132495abdb0b82bd99ef20a)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:28.830181 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f538ec-0c9f-40a7-9b48-f456df8f4c1a None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:28.831594 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 110/217] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:32:28 2022] GET /identity/v3/users/3dca63322132495abdb0b82bd99ef20a => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:28.837427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f538ec-0c9f-40a7-9b48-f456df8f4c1a None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:28.854071 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f9a64-7a23-4e5b-b48e-7dbaaf2b6e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:28.854784 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f9a64-7a23-4e5b-b48e-7dbaaf2b6e0e None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:28.855156 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f9a64-7a23-4e5b-b48e-7dbaaf2b6e0e None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:28.855487 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f9a64-7a23-4e5b-b48e-7dbaaf2b6e0e None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:28.856816 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125f9a64-7a23-4e5b-b48e-7dbaaf2b6e0e None admin] RBAC: Adding query filter params (name=reader) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:28.857221 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125f9a64-7a23-4e5b-b48e-7dbaaf2b6e0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:28.858693 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125f9a64-7a23-4e5b-b48e-7dbaaf2b6e0e None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:28.867226 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 108/218] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Oct 17 03:32:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:28.874650 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f9a64-7a23-4e5b-b48e-7dbaaf2b6e0e None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:28.943294 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:28.957779 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f538ec-0c9f-40a7-9b48-f456df8f4c1a None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:28.995263 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f9a64-7a23-4e5b-b48e-7dbaaf2b6e0e None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:28.997232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9aedc9-6fe6-4b16-a91e-7100ab9404f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:28.997865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9aedc9-6fe6-4b16-a91e-7100ab9404f5 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:28.998143 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9aedc9-6fe6-4b16-a91e-7100ab9404f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:28.998438 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9aedc9-6fe6-4b16-a91e-7100ab9404f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:28.999140 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9aedc9-6fe6-4b16-a91e-7100ab9404f5 None admin] RBAC: Adding query filter params (role.id=84a147decc7b417783086c1effe0fe20, user.id=3dca63322132495abdb0b82bd99ef20a, scope.system=all) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:28.999448 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9aedc9-6fe6-4b16-a91e-7100ab9404f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:29.001824 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9aedc9-6fe6-4b16-a91e-7100ab9404f5 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:29.009011 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 111/219] 10.222.0.17 () {56 vars in 1308 bytes} [Mon Oct 17 03:32:28 2022] GET /identity/v3/role_assignments?user.id=3dca63322132495abdb0b82bd99ef20a&scope.system=all&role.id=84a147decc7b417783086c1effe0fe20 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:29.041966 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485caa3b-8118-4731-befd-9136544c31d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:29.042556 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485caa3b-8118-4731-befd-9136544c31d9 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:29.042837 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485caa3b-8118-4731-befd-9136544c31d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:29.043112 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485caa3b-8118-4731-befd-9136544c31d9 None admin] PATH_INFO: `/v3/users/8235738370424e138c69ad6bd47b3119` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:29.045110 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485caa3b-8118-4731-befd-9136544c31d9 None admin] RBAC: Authorizing `identity:get_user(user_id=8235738370424e138c69ad6bd47b3119)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:29.046625 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485caa3b-8118-4731-befd-9136544c31d9 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:29.048826 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 109/220] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:32:28 2022] GET /identity/v3/users/8235738370424e138c69ad6bd47b3119 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:29.054584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9aedc9-6fe6-4b16-a91e-7100ab9404f5 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:29.094380 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:29.173652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9aedc9-6fe6-4b16-a91e-7100ab9404f5 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:29.210976 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c2cb3-fbba-4bae-b8ee-108abd54173d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:29.211953 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c2cb3-fbba-4bae-b8ee-108abd54173d None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:29.212449 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c2cb3-fbba-4bae-b8ee-108abd54173d None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:29.212911 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c2cb3-fbba-4bae-b8ee-108abd54173d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:29.285634 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c2cb3-fbba-4bae-b8ee-108abd54173d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:29.287435 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c2cb3-fbba-4bae-b8ee-108abd54173d None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:29.318800 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:29.396709 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 112/221] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:32:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 03:32:29.402435 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485caa3b-8118-4731-befd-9136544c31d9 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:29.510044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485caa3b-8118-4731-befd-9136544c31d9 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:29.535857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:29.546968 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee64ecc1-d455-4999-8918-7b87001f2ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:29.556996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64ecc1-d455-4999-8918-7b87001f2ede None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:29.556996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64ecc1-d455-4999-8918-7b87001f2ede None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:29.556996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee64ecc1-d455-4999-8918-7b87001f2ede None admin] PATH_INFO: `/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:29.556996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64ecc1-d455-4999-8918-7b87001f2ede None admin] RBAC: Authorizing `identity:get_project(project_id=9a1236f1bb0f4a499c0206128e5c97f9)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:29.556996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee64ecc1-d455-4999-8918-7b87001f2ede None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:29.556996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 110/222] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:32:29 2022] GET /identity/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:29.558666 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c2cb3-fbba-4bae-b8ee-108abd54173d None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:29.675712 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c2cb3-fbba-4bae-b8ee-108abd54173d None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:29.709504 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:29.714964 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e442975-2f8d-4a75-953b-b1033c3ed9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:29.715996 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e442975-2f8d-4a75-953b-b1033c3ed9b3 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:29.716278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e442975-2f8d-4a75-953b-b1033c3ed9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:29.716555 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e442975-2f8d-4a75-953b-b1033c3ed9b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:29.717293 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e442975-2f8d-4a75-953b-b1033c3ed9b3 None admin] RBAC: Adding query filter params (role.id=010a82aa9f3b4a23817fe234196cf259, scope.project.id=9a1236f1bb0f4a499c0206128e5c97f9, user.id=8235738370424e138c69ad6bd47b3119) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:29.717604 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e442975-2f8d-4a75-953b-b1033c3ed9b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:29.719133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e442975-2f8d-4a75-953b-b1033c3ed9b3 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:29.725597 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 113/223] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Oct 17 03:32:29 2022] GET /identity/v3/role_assignments?user.id=8235738370424e138c69ad6bd47b3119&scope.project.id=9a1236f1bb0f4a499c0206128e5c97f9&role.id=010a82aa9f3b4a23817fe234196cf259 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:30.980685 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a028967-e9ad-429b-a9d0-f247cbaab9ff None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:30.981187 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a028967-e9ad-429b-a9d0-f247cbaab9ff None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:30.981261 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a028967-e9ad-429b-a9d0-f247cbaab9ff None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:30.982771 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 111/224] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:30.987782 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd07ab7-2a0a-49e2-b0d5-d0e725099ff5 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:30.988171 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd07ab7-2a0a-49e2-b0d5-d0e725099ff5 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:30.988386 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd07ab7-2a0a-49e2-b0d5-d0e725099ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:31.006415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-8cd07ab7-2a0a-49e2-b0d5-d0e725099ff5 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:31.007626 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-8cd07ab7-2a0a-49e2-b0d5-d0e725099ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:31.025723 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfa69d-ffef-458c-960c-4d039d05d1f1 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:31.025999 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfa69d-ffef-458c-960c-4d039d05d1f1 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:31.026280 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfa69d-ffef-458c-960c-4d039d05d1f1 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:31.028183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 112/225] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:31.032857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a292953-7c41-4231-a5f7-1459f0226a2c None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:31.033143 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a292953-7c41-4231-a5f7-1459f0226a2c None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:31.033416 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a292953-7c41-4231-a5f7-1459f0226a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:31.052660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-0a292953-7c41-4231-a5f7-1459f0226a2c None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:31.054134 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-0a292953-7c41-4231-a5f7-1459f0226a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:31.079636 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 114/226] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:31.127682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 113/227] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 03:32:31.195404 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45903943-24b6-4e46-a493-44f836b4e44f None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:31.195713 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45903943-24b6-4e46-a493-44f836b4e44f None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:31.196010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45903943-24b6-4e46-a493-44f836b4e44f None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:31.197384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 115/228] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:31.203412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bf41f-29bf-446b-9167-9e1698bc5e18 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:31.203796 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bf41f-29bf-446b-9167-9e1698bc5e18 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:31.204394 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65bf41f-29bf-446b-9167-9e1698bc5e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:31.222986 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-e65bf41f-29bf-446b-9167-9e1698bc5e18 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:31.224180 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-e65bf41f-29bf-446b-9167-9e1698bc5e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:31.292226 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 114/229] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:31.348470 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45903943-24b6-4e46-a493-44f836b4e44f None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:31.390076 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bf41f-29bf-446b-9167-9e1698bc5e18 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:31.442059 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:31.457771 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45903943-24b6-4e46-a493-44f836b4e44f None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:31.486965 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:31.490667 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65bf41f-29bf-446b-9167-9e1698bc5e18 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:31.496263 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c98ea5-a770-4976-8b5d-3ca1e4ec8c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:31.497421 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c98ea5-a770-4976-8b5d-3ca1e4ec8c11 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:31.497698 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c98ea5-a770-4976-8b5d-3ca1e4ec8c11 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:31.497961 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c98ea5-a770-4976-8b5d-3ca1e4ec8c11 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:31.509442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c98ea5-a770-4976-8b5d-3ca1e4ec8c11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:31.514429 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-93c98ea5-a770-4976-8b5d-3ca1e4ec8c11 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 03:32:31.514429 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:31.514429 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:31.514429 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:31.514429 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:31.514429 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:31.514429 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:31.514429 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:31.514429 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:31.514429 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:31.514429 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:31.514429 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:31.514429 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:31.514429 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:31.514429 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:31.514429 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:31.515276 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:31.515276 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:31.515276 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:31.515276 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:31.515276 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:31.515276 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:31.515276 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:31.515276 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:31.515276 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:31.515276 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:31.515276 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:31.515276 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:31.515276 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:31.515276 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:31.515276 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:31.515276 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:31.520576 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:31.520576 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:31.520576 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:31.520576 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:31.520576 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:31.520576 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:31.520576 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:31.520576 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:31.520576 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:31.520576 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:31.520576 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 03:32:31.520576 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:31.520576 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 116/230] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:32:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:31.522056 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c98ea5-a770-4976-8b5d-3ca1e4ec8c11 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:31.532618 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96577ac6-0e9e-4973-ba63-e9479a4a0069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:31.533649 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96577ac6-0e9e-4973-ba63-e9479a4a0069 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:31.533943 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96577ac6-0e9e-4973-ba63-e9479a4a0069 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:31.534222 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96577ac6-0e9e-4973-ba63-e9479a4a0069 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:31.536104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96577ac6-0e9e-4973-ba63-e9479a4a0069 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:31.538147 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96577ac6-0e9e-4973-ba63-e9479a4a0069 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:31.540131 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 115/231] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:32:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:31.545974 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96577ac6-0e9e-4973-ba63-e9479a4a0069 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:31.627546 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c98ea5-a770-4976-8b5d-3ca1e4ec8c11 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:31.646542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96577ac6-0e9e-4973-ba63-e9479a4a0069 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:31.660189 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e6ca4-46e6-4bf0-9043-b31508527386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:31.660797 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e6ca4-46e6-4bf0-9043-b31508527386 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:31.661074 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e6ca4-46e6-4bf0-9043-b31508527386 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:31.661346 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e6ca4-46e6-4bf0-9043-b31508527386 None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:31.662009 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e6ca4-46e6-4bf0-9043-b31508527386 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:31.662311 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e6ca4-46e6-4bf0-9043-b31508527386 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:31.663656 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e6ca4-46e6-4bf0-9043-b31508527386 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:31.666003 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:31.672189 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 117/232] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Oct 17 03:32:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:31.675167 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e6ca4-46e6-4bf0-9043-b31508527386 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:31.684506 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87cee9-60d4-448a-b40a-912a233fd50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:31.685116 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87cee9-60d4-448a-b40a-912a233fd50e None admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:31.685413 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87cee9-60d4-448a-b40a-912a233fd50e None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:31.685680 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87cee9-60d4-448a-b40a-912a233fd50e None admin] PATH_INFO: `/v3/groups` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:31.686427 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b87cee9-60d4-448a-b40a-912a233fd50e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:31.687871 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b87cee9-60d4-448a-b40a-912a233fd50e None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:31.743259 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:31.749728 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 116/233] 10.222.0.17 () {60 vars in 1134 bytes} [Mon Oct 17 03:32:31 2022] POST /identity/v3/groups => generated 229 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:32:31.750548 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87cee9-60d4-448a-b40a-912a233fd50e None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:31.817804 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e6ca4-46e6-4bf0-9043-b31508527386 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:31.858290 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c9827-9875-44dc-8b4e-015beb863e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:31.859248 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c9827-9875-44dc-8b4e-015beb863e84 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:31.860005 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c9827-9875-44dc-8b4e-015beb863e84 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:31.860912 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c9827-9875-44dc-8b4e-015beb863e84 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:31.863291 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87cee9-60d4-448a-b40a-912a233fd50e None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:31.873788 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c9827-9875-44dc-8b4e-015beb863e84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:31.881331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-962c9827-9875-44dc-8b4e-015beb863e84 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 03:32:31.881331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:31.881331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:31.881331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:31.881331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:31.881331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:31.881331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:31.881331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:31.881331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:31.881331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:31.881331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:31.881331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:31.881331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:31.881331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:31.881331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:31.881331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:31.882387 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:31.882387 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:31.882387 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:31.882387 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:31.882387 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:31.882387 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:31.882387 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:31.882387 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:31.882387 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:31.882387 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:31.882387 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:31.882387 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:31.882387 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:31.882387 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:31.882387 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:31.882387 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:31.883328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:31.883328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:31.883328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:31.883328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:31.883328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:31.883328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:31.883328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:31.883328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:31.883328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:31.883328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:31.883328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 03:32:31.883328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:31.885768 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 118/234] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:32:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:31.891460 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c9827-9875-44dc-8b4e-015beb863e84 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:31.900740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0029c071-60ad-45f7-98e8-6f789c94a36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:31.901392 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029c071-60ad-45f7-98e8-6f789c94a36a None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:31.901671 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029c071-60ad-45f7-98e8-6f789c94a36a None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:31.901942 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029c071-60ad-45f7-98e8-6f789c94a36a None admin] PATH_INFO: `/v3/users/9d0a16b904824ce1940e89d6554bc4b1` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:31.920423 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0029c071-60ad-45f7-98e8-6f789c94a36a None admin] RBAC: Authorizing `identity:get_user(user_id=9d0a16b904824ce1940e89d6554bc4b1)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:31.921897 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0029c071-60ad-45f7-98e8-6f789c94a36a None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:31.923593 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 117/235] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:32:31 2022] GET /identity/v3/users/9d0a16b904824ce1940e89d6554bc4b1 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:31.929376 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0029c071-60ad-45f7-98e8-6f789c94a36a None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:31.980089 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:32.005621 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c9827-9875-44dc-8b4e-015beb863e84 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:32.015594 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:32.038465 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0029c071-60ad-45f7-98e8-6f789c94a36a None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:32.044625 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb190a6-d514-4b44-a779-077f04cafe6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:32.045248 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb190a6-d514-4b44-a779-077f04cafe6e None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:32.045529 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb190a6-d514-4b44-a779-077f04cafe6e None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:32.045802 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb190a6-d514-4b44-a779-077f04cafe6e None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:32.046475 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb190a6-d514-4b44-a779-077f04cafe6e None admin] RBAC: Adding query filter params (name=member) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:32.046810 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb190a6-d514-4b44-a779-077f04cafe6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:32.048621 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb190a6-d514-4b44-a779-077f04cafe6e None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:32.063515 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 119/236] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Oct 17 03:32:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:32.064673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb190a6-d514-4b44-a779-077f04cafe6e None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:32.078218 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed12c1-6b03-4756-b11d-4aca1e04573b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:32.078781 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed12c1-6b03-4756-b11d-4aca1e04573b None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:32.079055 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed12c1-6b03-4756-b11d-4aca1e04573b None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:32.079323 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed12c1-6b03-4756-b11d-4aca1e04573b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:32.080089 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed12c1-6b03-4756-b11d-4aca1e04573b None admin] RBAC: Adding query filter params (role.id=010a82aa9f3b4a23817fe234196cf259, user.id=9d0a16b904824ce1940e89d6554bc4b1, scope.system=all) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:32.080423 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed12c1-6b03-4756-b11d-4aca1e04573b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:32.081834 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed12c1-6b03-4756-b11d-4aca1e04573b None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:32.095786 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 118/237] 10.222.0.17 () {56 vars in 1308 bytes} [Mon Oct 17 03:32:31 2022] GET /identity/v3/role_assignments?user.id=9d0a16b904824ce1940e89d6554bc4b1&scope.system=all&role.id=010a82aa9f3b4a23817fe234196cf259 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:32.182225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb190a6-d514-4b44-a779-077f04cafe6e None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:32.218312 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ef485-23d5-4192-ae85-249e6c58652b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:32.218960 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ef485-23d5-4192-ae85-249e6c58652b None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:32.219237 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ef485-23d5-4192-ae85-249e6c58652b None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:32.219505 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ef485-23d5-4192-ae85-249e6c58652b None admin] PATH_INFO: `/v3/users/3dca63322132495abdb0b82bd99ef20a` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:32.220680 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ef485-23d5-4192-ae85-249e6c58652b None admin] RBAC: Authorizing `identity:get_user(user_id=3dca63322132495abdb0b82bd99ef20a)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:32.222119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ef485-23d5-4192-ae85-249e6c58652b None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:32.228702 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 120/238] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:32:32 2022] GET /identity/v3/users/3dca63322132495abdb0b82bd99ef20a => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:32.229503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed12c1-6b03-4756-b11d-4aca1e04573b None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:32.338967 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed12c1-6b03-4756-b11d-4aca1e04573b None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:32.341402 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:32.372798 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75788807-27e1-4640-abb2-951b7a0b72b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:32.373451 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75788807-27e1-4640-abb2-951b7a0b72b0 None admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:32.373727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75788807-27e1-4640-abb2-951b7a0b72b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:32.374002 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75788807-27e1-4640-abb2-951b7a0b72b0 None admin] PATH_INFO: `/v3/system/users/3dca63322132495abdb0b82bd99ef20a/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:32.375392 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75788807-27e1-4640-abb2-951b7a0b72b0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3dca63322132495abdb0b82bd99ef20a, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:32.376796 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75788807-27e1-4640-abb2-951b7a0b72b0 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:32.449047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 119/239] 10.222.0.17 () {58 vars in 1334 bytes} [Mon Oct 17 03:32:32 2022] PUT /identity/v3/system/users/3dca63322132495abdb0b82bd99ef20a/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:32:33.938590 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c31317-c5a2-473e-8118-2887768e3c30 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:33.939080 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c31317-c5a2-473e-8118-2887768e3c30 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:33.939080 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c31317-c5a2-473e-8118-2887768e3c30 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:33.940595 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 121/240] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:33.945596 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18844569-9718-400b-909e-3fe9c979371b None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:33.945977 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18844569-9718-400b-909e-3fe9c979371b None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:33.946152 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18844569-9718-400b-909e-3fe9c979371b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:33.964746 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-18844569-9718-400b-909e-3fe9c979371b None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:33.966734 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-18844569-9718-400b-909e-3fe9c979371b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:34.034121 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 120/241] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:34.209375 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182ddcc-0be9-4943-ba78-b1e254ac0da0 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:34.209661 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182ddcc-0be9-4943-ba78-b1e254ac0da0 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:34.209852 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182ddcc-0be9-4943-ba78-b1e254ac0da0 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:34.211334 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 122/242] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:34.216540 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87616d2-c595-4f36-a8ff-58600f24451e None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:34.216828 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87616d2-c595-4f36-a8ff-58600f24451e None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:34.217099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87616d2-c595-4f36-a8ff-58600f24451e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:34.234820 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-a87616d2-c595-4f36-a8ff-58600f24451e None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:34.236009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-a87616d2-c595-4f36-a8ff-58600f24451e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:34.301591 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 121/243] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:34.302039 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182ddcc-0be9-4943-ba78-b1e254ac0da0 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:34.348555 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:34.408777 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182ddcc-0be9-4943-ba78-b1e254ac0da0 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:34.442562 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8567eff-d2b9-4014-bb90-7a075332dac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:34.443193 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8567eff-d2b9-4014-bb90-7a075332dac5 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:34.443474 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8567eff-d2b9-4014-bb90-7a075332dac5 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:34.443761 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8567eff-d2b9-4014-bb90-7a075332dac5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:34.444862 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8567eff-d2b9-4014-bb90-7a075332dac5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:34.446283 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8567eff-d2b9-4014-bb90-7a075332dac5 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:34.451766 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 123/244] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:32:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:34.454112 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87616d2-c595-4f36-a8ff-58600f24451e None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:34.486216 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:34.561493 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87616d2-c595-4f36-a8ff-58600f24451e None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:34.571678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8567eff-d2b9-4014-bb90-7a075332dac5 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:34.600892 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df55c42-4e15-464a-929a-569067dd92a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:34.601496 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df55c42-4e15-464a-929a-569067dd92a9 None admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:34.601776 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df55c42-4e15-464a-929a-569067dd92a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:34.602044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df55c42-4e15-464a-929a-569067dd92a9 None admin] PATH_INFO: `/v3/groups` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:34.602675 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df55c42-4e15-464a-929a-569067dd92a9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:34.604116 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df55c42-4e15-464a-929a-569067dd92a9 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:34.670668 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8567eff-d2b9-4014-bb90-7a075332dac5 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:34.680088 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:34.681028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 122/245] 10.222.0.17 () {60 vars in 1134 bytes} [Mon Oct 17 03:32:34 2022] POST /identity/v3/groups => generated 226 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:32:34.682169 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b89de-afc3-44e1-8773-7edd4bb0dbe7 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:34.682460 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b89de-afc3-44e1-8773-7edd4bb0dbe7 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:34.682739 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1b89de-afc3-44e1-8773-7edd4bb0dbe7 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:34.684654 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 123/246] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:34.693486 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547ce80-9346-47f4-8488-f0f411f5fe09 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:34.693791 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547ce80-9346-47f4-8488-f0f411f5fe09 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:34.694043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7547ce80-9346-47f4-8488-f0f411f5fe09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:34.713819 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-7547ce80-9346-47f4-8488-f0f411f5fe09 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:34.715062 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-7547ce80-9346-47f4-8488-f0f411f5fe09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:34.720801 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d5b18-4c5c-4173-a746-2d6570949657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:34.721689 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d5b18-4c5c-4173-a746-2d6570949657 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:34.721974 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d5b18-4c5c-4173-a746-2d6570949657 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:34.722302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d5b18-4c5c-4173-a746-2d6570949657 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:34.735694 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45d5b18-4c5c-4173-a746-2d6570949657 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:34.741071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-f45d5b18-4c5c-4173-a746-2d6570949657 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 03:32:34.741071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:34.741071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:34.741071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:34.741071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:34.741071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:34.741071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:34.741071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:34.741071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:34.741071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:34.741071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:34.741071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:34.741071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:34.741071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:34.741071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:34.741071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:34.742182 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:34.742182 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:34.742182 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:34.742182 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:34.742182 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:34.742182 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:34.742182 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:34.742182 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:34.742182 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:34.742182 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:34.742182 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:34.742182 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:34.742182 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:34.742182 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:34.742182 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:34.742182 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:34.742955 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:34.742955 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:34.742955 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:34.742955 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:34.742955 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:34.742955 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:34.742955 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:34.742955 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:34.742955 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:34.742955 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:34.742955 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 03:32:34.742955 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:34.742955 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 124/247] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:32:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:32:34.750800 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d5b18-4c5c-4173-a746-2d6570949657 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:34.807934 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 124/248] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 03:32:34.870490 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45d5b18-4c5c-4173-a746-2d6570949657 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:34.907024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756d95d-396b-481f-bd24-e3f5db1e411d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:34.907915 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756d95d-396b-481f-bd24-e3f5db1e411d None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:34.908332 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756d95d-396b-481f-bd24-e3f5db1e411d None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:34.908739 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6756d95d-396b-481f-bd24-e3f5db1e411d None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:34.909539 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6756d95d-396b-481f-bd24-e3f5db1e411d None admin] RBAC: Adding query filter params (name=reader) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:34.910018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6756d95d-396b-481f-bd24-e3f5db1e411d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:34.911497 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6756d95d-396b-481f-bd24-e3f5db1e411d None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:34.920843 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 125/249] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Oct 17 03:32:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:34.926262 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547ce80-9346-47f4-8488-f0f411f5fe09 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:34.944991 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:35.038588 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7547ce80-9346-47f4-8488-f0f411f5fe09 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:35.074664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756d95d-396b-481f-bd24-e3f5db1e411d None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:35.076906 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c06c66-1af6-40b1-9ffd-cb8ce84ffd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:35.077683 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c06c66-1af6-40b1-9ffd-cb8ce84ffd00 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:35.077973 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c06c66-1af6-40b1-9ffd-cb8ce84ffd00 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:35.078251 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c06c66-1af6-40b1-9ffd-cb8ce84ffd00 None admin] PATH_INFO: `/v3/users/9d0a16b904824ce1940e89d6554bc4b1` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:35.079548 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c06c66-1af6-40b1-9ffd-cb8ce84ffd00 None admin] RBAC: Authorizing `identity:get_user(user_id=9d0a16b904824ce1940e89d6554bc4b1)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:35.081041 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c06c66-1af6-40b1-9ffd-cb8ce84ffd00 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:35.087342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 125/250] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:32:34 2022] GET /identity/v3/users/9d0a16b904824ce1940e89d6554bc4b1 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:35.088649 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c06c66-1af6-40b1-9ffd-cb8ce84ffd00 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:35.118271 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:35.204254 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:35.210390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6756d95d-396b-481f-bd24-e3f5db1e411d None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:35.239577 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c06c66-1af6-40b1-9ffd-cb8ce84ffd00 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:35.252165 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac32a7d-684c-4232-92ad-95ec347bb8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:35.252815 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac32a7d-684c-4232-92ad-95ec347bb8f0 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:35.253199 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac32a7d-684c-4232-92ad-95ec347bb8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:35.253484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac32a7d-684c-4232-92ad-95ec347bb8f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:35.265954 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac32a7d-684c-4232-92ad-95ec347bb8f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:35.271390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-4ac32a7d-684c-4232-92ad-95ec347bb8f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 03:32:35.271390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:35.271390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:35.271390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:35.271390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:35.271390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:35.271390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:35.271390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:35.271390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:35.271390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:35.271390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:35.271390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:35.271390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:35.271390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:35.271390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:35.271390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:35.272342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:35.272342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:35.272342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:35.272342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:35.272342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:35.272342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:35.272342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:35.272342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:35.272342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:35.272342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:35.272342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:35.272342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:35.272342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:35.272342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:35.272342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:35.272342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:35.273103 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:35.273103 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:35.273103 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:35.273103 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:35.273103 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:35.273103 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:35.273103 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:35.273103 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:35.273103 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:35.273103 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:35.273103 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 03:32:35.273103 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:35.273103 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 126/251] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:32:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:35.282617 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac32a7d-684c-4232-92ad-95ec347bb8f0 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:35.288011 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5830b-caa9-4877-b74c-5ff1411372ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:35.288969 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5830b-caa9-4877-b74c-5ff1411372ab None admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:35.289444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5830b-caa9-4877-b74c-5ff1411372ab None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:35.289926 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5830b-caa9-4877-b74c-5ff1411372ab None admin] PATH_INFO: `/v3/system/users/9d0a16b904824ce1940e89d6554bc4b1/roles/010a82aa9f3b4a23817fe234196cf259` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:35.291571 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5830b-caa9-4877-b74c-5ff1411372ab None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9d0a16b904824ce1940e89d6554bc4b1, role_id=010a82aa9f3b4a23817fe234196cf259)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:35.293215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5830b-caa9-4877-b74c-5ff1411372ab None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:35.352796 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 126/252] 10.222.0.17 () {58 vars in 1334 bytes} [Mon Oct 17 03:32:35 2022] PUT /identity/v3/system/users/9d0a16b904824ce1940e89d6554bc4b1/roles/010a82aa9f3b4a23817fe234196cf259 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:32:35.389140 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:35.403770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac32a7d-684c-4232-92ad-95ec347bb8f0 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:35.447421 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4cf64-0f73-412c-a20e-6982d7f19be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:35.448685 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4cf64-0f73-412c-a20e-6982d7f19be7 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:35.449880 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4cf64-0f73-412c-a20e-6982d7f19be7 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:35.450165 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4cf64-0f73-412c-a20e-6982d7f19be7 None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:35.450919 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d4cf64-0f73-412c-a20e-6982d7f19be7 None admin] RBAC: Adding query filter params (name=member) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:35.451891 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d4cf64-0f73-412c-a20e-6982d7f19be7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:35.453294 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d4cf64-0f73-412c-a20e-6982d7f19be7 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:35.463377 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 127/253] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Oct 17 03:32:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:35.469909 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5830b-caa9-4877-b74c-5ff1411372ab None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:35.600194 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5830b-caa9-4877-b74c-5ff1411372ab None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:35.639235 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff47f0-9883-4991-b451-bf1050431a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:35.639926 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff47f0-9883-4991-b451-bf1050431a17 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:35.640214 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff47f0-9883-4991-b451-bf1050431a17 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:35.640490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff47f0-9883-4991-b451-bf1050431a17 None admin] PATH_INFO: `/v3/users/3dca63322132495abdb0b82bd99ef20a` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:35.643295 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff47f0-9883-4991-b451-bf1050431a17 None admin] RBAC: Authorizing `identity:get_user(user_id=3dca63322132495abdb0b82bd99ef20a)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:35.643295 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff47f0-9883-4991-b451-bf1050431a17 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:35.645093 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 127/254] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:32:35 2022] GET /identity/v3/users/3dca63322132495abdb0b82bd99ef20a => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:35.651047 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4cf64-0f73-412c-a20e-6982d7f19be7 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:35.787610 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4cf64-0f73-412c-a20e-6982d7f19be7 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:35.831690 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e59b751-50cc-4170-a3f5-9ec8bad09b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:35.832673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59b751-50cc-4170-a3f5-9ec8bad09b86 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:35.833144 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59b751-50cc-4170-a3f5-9ec8bad09b86 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:35.833606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e59b751-50cc-4170-a3f5-9ec8bad09b86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:35.834544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e59b751-50cc-4170-a3f5-9ec8bad09b86 None admin] RBAC: Adding query filter params (role.id=84a147decc7b417783086c1effe0fe20, user.id=3dca63322132495abdb0b82bd99ef20a, scope.system=all) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:35.835059 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e59b751-50cc-4170-a3f5-9ec8bad09b86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:35.836745 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e59b751-50cc-4170-a3f5-9ec8bad09b86 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:35.850354 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 128/255] 10.222.0.17 () {56 vars in 1308 bytes} [Mon Oct 17 03:32:35 2022] GET /identity/v3/role_assignments?user.id=3dca63322132495abdb0b82bd99ef20a&scope.system=all&role.id=84a147decc7b417783086c1effe0fe20 => generated 510 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:37.169182 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790f7a2-9874-4be1-a1b2-bb61d7f81648 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:37.170287 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790f7a2-9874-4be1-a1b2-bb61d7f81648 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:37.170778 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790f7a2-9874-4be1-a1b2-bb61d7f81648 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:37.172684 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 128/256] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:37.181215 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8636e-5a97-45d9-bad8-467516851523 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:37.182166 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8636e-5a97-45d9-bad8-467516851523 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:37.182658 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8636e-5a97-45d9-bad8-467516851523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:37.209768 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-0cb8636e-5a97-45d9-bad8-467516851523 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:37.211489 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-0cb8636e-5a97-45d9-bad8-467516851523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:37.218201 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:37.289415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 129/257] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 03:32:37.527855 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0389a45-8c24-48a1-973c-4c403c82586d None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:37.528178 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0389a45-8c24-48a1-973c-4c403c82586d None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:37.528454 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0389a45-8c24-48a1-973c-4c403c82586d None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:37.530118 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 129/258] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:37.535062 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7cb47-aeb4-47e7-9dd4-a06e7e57faa6 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:37.535362 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7cb47-aeb4-47e7-9dd4-a06e7e57faa6 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:37.535642 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7cb47-aeb4-47e7-9dd4-a06e7e57faa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:37.557801 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-f1e7cb47-aeb4-47e7-9dd4-a06e7e57faa6 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:37.559552 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-f1e7cb47-aeb4-47e7-9dd4-a06e7e57faa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:37.574768 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 130/259] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 03:32:37.837659 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0389a45-8c24-48a1-973c-4c403c82586d None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:37.852260 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0389a45-8c24-48a1-973c-4c403c82586d None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:37.858285 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64341a42-eb80-4452-91c9-fd738381e5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:37.858917 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64341a42-eb80-4452-91c9-fd738381e5d5 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:37.859200 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64341a42-eb80-4452-91c9-fd738381e5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:37.859478 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64341a42-eb80-4452-91c9-fd738381e5d5 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:37.875319 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64341a42-eb80-4452-91c9-fd738381e5d5 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:37.880425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-64341a42-eb80-4452-91c9-fd738381e5d5 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 03:32:37.880425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:37.880425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:37.880425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:37.880425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:37.880425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:37.880425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:37.880425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:37.880425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:37.880425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:37.880425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:37.880425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:37.880425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:37.880425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:37.880425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:37.880425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:37.881269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:37.881269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:37.881269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:37.881269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:37.881269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:37.881269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:37.881269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:37.881269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:37.881269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:37.881269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:37.881269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:37.881269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:37.881269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:37.881269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:37.881269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:37.881269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:37.882023 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:37.882023 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:37.882023 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:37.882023 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:37.882023 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:37.882023 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:37.882023 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:37.882023 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:37.882023 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:37.882023 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:37.882023 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 03:32:37.882023 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:37.883057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 130/260] 10.222.0.17 () {56 vars in 1120 bytes} [Mon Oct 17 03:32:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 45 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:37.887856 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7cb47-aeb4-47e7-9dd4-a06e7e57faa6 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:37.898768 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7cb47-aeb4-47e7-9dd4-a06e7e57faa6 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:37.906130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5aec2-33ed-45da-ad94-1b353ed548a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:37.906841 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5aec2-33ed-45da-ad94-1b353ed548a0 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:37.907129 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5aec2-33ed-45da-ad94-1b353ed548a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:37.907403 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5aec2-33ed-45da-ad94-1b353ed548a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:37.908119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5aec2-33ed-45da-ad94-1b353ed548a0 admin admin] RBAC: Adding query filter params (name=member) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:37.908431 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5aec2-33ed-45da-ad94-1b353ed548a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:37.910280 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: WARNING py.warnings [None req-17a5aec2-33ed-45da-ad94-1b353ed548a0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 17 03:32:37.910280 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: warnings.warn(msg) Oct 17 03:32:37.910280 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: Oct 17 03:32:37.911270 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5aec2-33ed-45da-ad94-1b353ed548a0 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:37.923442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 131/261] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Oct 17 03:32:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:37.929155 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64341a42-eb80-4452-91c9-fd738381e5d5 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:37.935812 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:37.939893 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64341a42-eb80-4452-91c9-fd738381e5d5 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:37.949093 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49bc4e-7c0c-4acc-b08e-a324ae0e3fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:37.949738 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49bc4e-7c0c-4acc-b08e-a324ae0e3fc6 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:37.950024 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49bc4e-7c0c-4acc-b08e-a324ae0e3fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:37.950437 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49bc4e-7c0c-4acc-b08e-a324ae0e3fc6 admin admin] PATH_INFO: `/v3/groups/32be849bef724c94a2e14a0e8b62b35d` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:37.957954 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d49bc4e-7c0c-4acc-b08e-a324ae0e3fc6 admin admin] RBAC: Authorizing `identity:get_group(group_id=32be849bef724c94a2e14a0e8b62b35d)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:37.962607 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: WARNING py.warnings [None req-8d49bc4e-7c0c-4acc-b08e-a324ae0e3fc6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 03:32:37.962607 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: warnings.warn(msg) Oct 17 03:32:37.962607 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: Oct 17 03:32:37.962607 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d49bc4e-7c0c-4acc-b08e-a324ae0e3fc6 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:37.963105 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 131/262] 10.222.0.17 () {56 vars in 1201 bytes} [Mon Oct 17 03:32:37 2022] GET /identity/v3/groups/32be849bef724c94a2e14a0e8b62b35d => generated 226 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:37.969687 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5aec2-33ed-45da-ad94-1b353ed548a0 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:37.981254 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5aec2-33ed-45da-ad94-1b353ed548a0 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:37.987066 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165907b1-4e9d-44bb-9f2f-4963344af3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:37.987863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165907b1-4e9d-44bb-9f2f-4963344af3e3 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:37.988152 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165907b1-4e9d-44bb-9f2f-4963344af3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:37.988429 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165907b1-4e9d-44bb-9f2f-4963344af3e3 admin admin] PATH_INFO: `/v3/projects/bc8197db3d20408990645225920f63d7` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:37.989625 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165907b1-4e9d-44bb-9f2f-4963344af3e3 admin admin] RBAC: Authorizing `identity:get_project(project_id=bc8197db3d20408990645225920f63d7)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:37.991889 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165907b1-4e9d-44bb-9f2f-4963344af3e3 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:37.994926 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 132/263] 10.222.0.17 () {56 vars in 1207 bytes} [Mon Oct 17 03:32:37 2022] GET /identity/v3/projects/bc8197db3d20408990645225920f63d7 => generated 298 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:38.000043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49bc4e-7c0c-4acc-b08e-a324ae0e3fc6 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:38.010995 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49bc4e-7c0c-4acc-b08e-a324ae0e3fc6 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:38.018355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a535753-ed80-4522-9f5d-965b3f745832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:38.019037 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a535753-ed80-4522-9f5d-965b3f745832 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:38.019321 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a535753-ed80-4522-9f5d-965b3f745832 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:38.019649 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a535753-ed80-4522-9f5d-965b3f745832 admin admin] PATH_INFO: `/v3/projects/bc8197db3d20408990645225920f63d7/groups/32be849bef724c94a2e14a0e8b62b35d/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:38.022654 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a535753-ed80-4522-9f5d-965b3f745832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc8197db3d20408990645225920f63d7, group_id=32be849bef724c94a2e14a0e8b62b35d, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:38.023809 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: WARNING py.warnings [None req-9a535753-ed80-4522-9f5d-965b3f745832 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 03:32:38.023809 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: warnings.warn(msg) Oct 17 03:32:38.023809 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: Oct 17 03:32:38.025846 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a535753-ed80-4522-9f5d-965b3f745832 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:38.104430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 132/264] 10.222.0.17 () {58 vars in 1463 bytes} [Mon Oct 17 03:32:37 2022] PUT /identity/v3/projects/bc8197db3d20408990645225920f63d7/groups/32be849bef724c94a2e14a0e8b62b35d/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:32:38.217497 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165907b1-4e9d-44bb-9f2f-4963344af3e3 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:38.228205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165907b1-4e9d-44bb-9f2f-4963344af3e3 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:38.290095 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c840cb0-9838-4efb-82f6-8188deb6f4ac None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:38.290625 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c840cb0-9838-4efb-82f6-8188deb6f4ac None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:38.290698 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c840cb0-9838-4efb-82f6-8188deb6f4ac None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:38.292245 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 133/265] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:38.303359 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b7c9d-5a01-4e5d-aff8-2f4b83f13090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:38.304176 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b7c9d-5a01-4e5d-aff8-2f4b83f13090 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:38.304459 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b7c9d-5a01-4e5d-aff8-2f4b83f13090 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:38.304738 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b7c9d-5a01-4e5d-aff8-2f4b83f13090 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:38.305286 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8eec7-d335-44ff-8e89-8aa4082fb713 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:38.305582 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8eec7-d335-44ff-8e89-8aa4082fb713 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:38.305860 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a8eec7-d335-44ff-8e89-8aa4082fb713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:38.316799 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7b7c9d-5a01-4e5d-aff8-2f4b83f13090 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:38.322486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-7d7b7c9d-5a01-4e5d-aff8-2f4b83f13090 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 03:32:38.322486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:38.322486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:38.322486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:38.322486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:38.322486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:38.322486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:38.322486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:38.322486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:38.322486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:38.322486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:38.322486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:38.322486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:38.322486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:38.322486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:38.322486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:38.323418 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:38.323418 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:38.323418 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:38.323418 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:38.323418 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:38.323418 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:38.323418 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:38.323418 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:38.323418 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:38.323418 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:38.323418 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:38.323418 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:38.323418 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:38.323418 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:38.323418 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:38.323418 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:38.324770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:38.324770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:38.324770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:38.324770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:38.324770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:38.324770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:38.324770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:38.324770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:38.324770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:38.324770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:38.324770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 03:32:38.324770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:38.324770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 133/266] 10.222.0.17 () {56 vars in 1135 bytes} [Mon Oct 17 03:32:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 107 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:38.332268 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b7c9d-5a01-4e5d-aff8-2f4b83f13090 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:38.346587 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b7c9d-5a01-4e5d-aff8-2f4b83f13090 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:38.348769 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-70a8eec7-d335-44ff-8e89-8aa4082fb713 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:38.349963 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-70a8eec7-d335-44ff-8e89-8aa4082fb713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:38.352870 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d310614-48e2-4658-8328-13961893be82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:38.353606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d310614-48e2-4658-8328-13961893be82 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:38.353894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d310614-48e2-4658-8328-13961893be82 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:38.354283 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d310614-48e2-4658-8328-13961893be82 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:38.355084 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d310614-48e2-4658-8328-13961893be82 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:38.355390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d310614-48e2-4658-8328-13961893be82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:38.357152 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d310614-48e2-4658-8328-13961893be82 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:38.371656 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 134/267] 10.222.0.17 () {56 vars in 1132 bytes} [Mon Oct 17 03:32:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:38.377811 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d310614-48e2-4658-8328-13961893be82 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:38.378095 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 134/268] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 17 03:32:38.393308 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d310614-48e2-4658-8328-13961893be82 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:38.399550 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c184476-60cc-4148-aa7c-b4fd3af323f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:38.400285 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c184476-60cc-4148-aa7c-b4fd3af323f7 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:38.400569 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c184476-60cc-4148-aa7c-b4fd3af323f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:38.400844 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c184476-60cc-4148-aa7c-b4fd3af323f7 admin admin] PATH_INFO: `/v3/groups/32be849bef724c94a2e14a0e8b62b35d` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:38.402868 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c184476-60cc-4148-aa7c-b4fd3af323f7 admin admin] RBAC: Authorizing `identity:get_group(group_id=32be849bef724c94a2e14a0e8b62b35d)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:38.403884 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: WARNING py.warnings [None req-4c184476-60cc-4148-aa7c-b4fd3af323f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 03:32:38.403884 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: warnings.warn(msg) Oct 17 03:32:38.403884 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: Oct 17 03:32:38.405217 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c184476-60cc-4148-aa7c-b4fd3af323f7 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:38.411448 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 135/269] 10.222.0.17 () {56 vars in 1201 bytes} [Mon Oct 17 03:32:38 2022] GET /identity/v3/groups/32be849bef724c94a2e14a0e8b62b35d => generated 226 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:38.413350 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8eec7-d335-44ff-8e89-8aa4082fb713 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:38.425695 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a8eec7-d335-44ff-8e89-8aa4082fb713 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:38.435420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0a2a0-3780-4d38-bcaf-ba0772dc8ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:38.436509 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0a2a0-3780-4d38-bcaf-ba0772dc8ae2 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:38.437038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0a2a0-3780-4d38-bcaf-ba0772dc8ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:38.437543 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca0a2a0-3780-4d38-bcaf-ba0772dc8ae2 admin admin] PATH_INFO: `/v3/projects/bc8197db3d20408990645225920f63d7` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:38.438912 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0a2a0-3780-4d38-bcaf-ba0772dc8ae2 admin admin] RBAC: Authorizing `identity:get_project(project_id=bc8197db3d20408990645225920f63d7)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:38.441331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca0a2a0-3780-4d38-bcaf-ba0772dc8ae2 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:38.444652 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 135/270] 10.222.0.17 () {56 vars in 1207 bytes} [Mon Oct 17 03:32:38 2022] GET /identity/v3/projects/bc8197db3d20408990645225920f63d7 => generated 298 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:38.450219 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c184476-60cc-4148-aa7c-b4fd3af323f7 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:38.461075 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c184476-60cc-4148-aa7c-b4fd3af323f7 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:38.469006 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababf49a-f49a-46ed-8793-29172c888e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:38.470069 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababf49a-f49a-46ed-8793-29172c888e95 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:38.470555 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababf49a-f49a-46ed-8793-29172c888e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:38.471019 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababf49a-f49a-46ed-8793-29172c888e95 admin admin] PATH_INFO: `/v3/projects/bc8197db3d20408990645225920f63d7/groups/32be849bef724c94a2e14a0e8b62b35d/roles/609a4bf154ff4dcabb408b3789d28983` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:38.473622 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababf49a-f49a-46ed-8793-29172c888e95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc8197db3d20408990645225920f63d7, group_id=32be849bef724c94a2e14a0e8b62b35d, role_id=609a4bf154ff4dcabb408b3789d28983)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:38.474883 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: WARNING py.warnings [None req-ababf49a-f49a-46ed-8793-29172c888e95 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 03:32:38.474883 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: warnings.warn(msg) Oct 17 03:32:38.474883 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: Oct 17 03:32:38.477385 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababf49a-f49a-46ed-8793-29172c888e95 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:38.545612 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 136/271] 10.222.0.17 () {58 vars in 1463 bytes} [Mon Oct 17 03:32:38 2022] PUT /identity/v3/projects/bc8197db3d20408990645225920f63d7/groups/32be849bef724c94a2e14a0e8b62b35d/roles/609a4bf154ff4dcabb408b3789d28983 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:32:38.716237 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0a2a0-3780-4d38-bcaf-ba0772dc8ae2 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:38.727046 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca0a2a0-3780-4d38-bcaf-ba0772dc8ae2 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:38.804154 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43e358-0f20-41cc-9614-19f46f53e4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:38.805170 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43e358-0f20-41cc-9614-19f46f53e4e9 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:38.805644 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43e358-0f20-41cc-9614-19f46f53e4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:38.806063 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43e358-0f20-41cc-9614-19f46f53e4e9 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:38.820458 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43e358-0f20-41cc-9614-19f46f53e4e9 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:38.826274 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-5a43e358-0f20-41cc-9614-19f46f53e4e9 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 03:32:38.826274 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:38.826274 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:38.826274 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:38.826274 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:38.826274 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:38.826274 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:38.826274 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:38.826274 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:38.826274 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:38.826274 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:38.826274 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:38.826274 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:38.826274 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:38.826274 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:38.826274 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:38.827368 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:38.827368 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:38.827368 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:38.827368 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:38.827368 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:38.827368 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:38.827368 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:38.827368 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:38.827368 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:38.827368 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:38.827368 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:38.827368 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:38.827368 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:38.827368 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:38.827368 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:38.827368 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:38.828392 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:38.828392 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:38.828392 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:38.828392 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:38.828392 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:38.828392 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:38.828392 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:38.828392 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:38.828392 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:38.828392 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:38.828392 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 03:32:38.828392 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:38.833609 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 136/272] 10.222.0.17 () {56 vars in 1120 bytes} [Mon Oct 17 03:32:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 115 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:38.836756 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababf49a-f49a-46ed-8793-29172c888e95 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:38.849646 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababf49a-f49a-46ed-8793-29172c888e95 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:38.854734 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14fa41-049e-4c59-8e0a-8d8706f4fb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:38.855877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14fa41-049e-4c59-8e0a-8d8706f4fb9d admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:38.856330 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14fa41-049e-4c59-8e0a-8d8706f4fb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:38.856783 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14fa41-049e-4c59-8e0a-8d8706f4fb9d admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:38.857647 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14fa41-049e-4c59-8e0a-8d8706f4fb9d admin admin] RBAC: Adding query filter params (name=member) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:38.858108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14fa41-049e-4c59-8e0a-8d8706f4fb9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:38.860168 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14fa41-049e-4c59-8e0a-8d8706f4fb9d admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:38.873287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 137/273] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Oct 17 03:32:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:38.878970 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43e358-0f20-41cc-9614-19f46f53e4e9 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:38.891477 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43e358-0f20-41cc-9614-19f46f53e4e9 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:38.899812 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc08d1-b31f-4c67-8aa5-33e9013f24f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:38.900774 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc08d1-b31f-4c67-8aa5-33e9013f24f9 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:38.901205 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc08d1-b31f-4c67-8aa5-33e9013f24f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:38.901658 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc08d1-b31f-4c67-8aa5-33e9013f24f9 admin admin] PATH_INFO: `/v3/groups/32be849bef724c94a2e14a0e8b62b35d` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:38.903076 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcc08d1-b31f-4c67-8aa5-33e9013f24f9 admin admin] RBAC: Authorizing `identity:get_group(group_id=32be849bef724c94a2e14a0e8b62b35d)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:38.905304 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcc08d1-b31f-4c67-8aa5-33e9013f24f9 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:38.908270 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 137/274] 10.222.0.17 () {56 vars in 1201 bytes} [Mon Oct 17 03:32:38 2022] GET /identity/v3/groups/32be849bef724c94a2e14a0e8b62b35d => generated 226 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:38.912773 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14fa41-049e-4c59-8e0a-8d8706f4fb9d admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:38.923892 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14fa41-049e-4c59-8e0a-8d8706f4fb9d admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:38.928978 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea33ce51-30df-4b96-aa09-fab64eafc95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:38.929856 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33ce51-30df-4b96-aa09-fab64eafc95c admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:38.930315 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33ce51-30df-4b96-aa09-fab64eafc95c admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:38.930732 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33ce51-30df-4b96-aa09-fab64eafc95c admin admin] PATH_INFO: `/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:38.932135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea33ce51-30df-4b96-aa09-fab64eafc95c admin admin] RBAC: Authorizing `identity:get_project(project_id=9a1236f1bb0f4a499c0206128e5c97f9)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:38.935113 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea33ce51-30df-4b96-aa09-fab64eafc95c admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:38.937385 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 138/275] 10.222.0.17 () {56 vars in 1207 bytes} [Mon Oct 17 03:32:38 2022] GET /identity/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9 => generated 302 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:38.942684 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc08d1-b31f-4c67-8aa5-33e9013f24f9 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:38.954348 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc08d1-b31f-4c67-8aa5-33e9013f24f9 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:38.963769 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053e44e5-9eba-42ea-bd1a-34a18c6685ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:38.964765 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e44e5-9eba-42ea-bd1a-34a18c6685ec admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:38.965519 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e44e5-9eba-42ea-bd1a-34a18c6685ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:38.965943 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e44e5-9eba-42ea-bd1a-34a18c6685ec admin admin] PATH_INFO: `/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9/groups/32be849bef724c94a2e14a0e8b62b35d/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:38.968243 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053e44e5-9eba-42ea-bd1a-34a18c6685ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a1236f1bb0f4a499c0206128e5c97f9, group_id=32be849bef724c94a2e14a0e8b62b35d, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:38.971250 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053e44e5-9eba-42ea-bd1a-34a18c6685ec admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:39.027231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57513c3-261a-4919-897b-11f9aa20367f None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:39.027771 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57513c3-261a-4919-897b-11f9aa20367f None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:39.028196 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57513c3-261a-4919-897b-11f9aa20367f None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:39.029799 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 139/276] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:39.040313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512206e7-6585-4fa1-ae05-1ab345a78f01 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:39.040872 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512206e7-6585-4fa1-ae05-1ab345a78f01 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:39.041299 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512206e7-6585-4fa1-ae05-1ab345a78f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:39.064880 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-512206e7-6585-4fa1-ae05-1ab345a78f01 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:39.066378 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-512206e7-6585-4fa1-ae05-1ab345a78f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:39.077577 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 138/277] 10.222.0.17 () {58 vars in 1463 bytes} [Mon Oct 17 03:32:38 2022] PUT /identity/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9/groups/32be849bef724c94a2e14a0e8b62b35d/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:32:39.112632 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:39.160626 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 140/278] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 03:32:39.706796 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053e44e5-9eba-42ea-bd1a-34a18c6685ec admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:39.728455 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be43efb-9b13-4aa4-a175-0b710a71b071 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:39.728985 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be43efb-9b13-4aa4-a175-0b710a71b071 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:39.728985 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be43efb-9b13-4aa4-a175-0b710a71b071 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:39.732077 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 141/279] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:39.737658 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1ec3c-f3b0-49f3-a6a9-6cc4e5470b60 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:39.737946 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1ec3c-f3b0-49f3-a6a9-6cc4e5470b60 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:39.738225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1ec3c-f3b0-49f3-a6a9-6cc4e5470b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:39.776077 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-5cb1ec3c-f3b0-49f3-a6a9-6cc4e5470b60 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:39.777991 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-5cb1ec3c-f3b0-49f3-a6a9-6cc4e5470b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:39.865970 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:39.870562 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053e44e5-9eba-42ea-bd1a-34a18c6685ec admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:39.884596 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 142/280] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 17 03:32:39.886832 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa36c0-4470-48fe-87b3-de02b0567bae None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:39.886832 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa36c0-4470-48fe-87b3-de02b0567bae None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:39.887105 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa36c0-4470-48fe-87b3-de02b0567bae None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:39.888700 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 143/281] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:39.895225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da6ac2-3afd-4394-bb0d-84063c58ebe8 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:39.895581 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da6ac2-3afd-4394-bb0d-84063c58ebe8 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:39.895887 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68da6ac2-3afd-4394-bb0d-84063c58ebe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:39.924096 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2918b0-8800-41d2-84ea-2813ec4a119c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:39.924752 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2918b0-8800-41d2-84ea-2813ec4a119c None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:39.925037 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2918b0-8800-41d2-84ea-2813ec4a119c None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:39.925319 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2918b0-8800-41d2-84ea-2813ec4a119c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:39.932836 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-68da6ac2-3afd-4394-bb0d-84063c58ebe8 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:39.934017 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-68da6ac2-3afd-4394-bb0d-84063c58ebe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:39.942611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2918b0-8800-41d2-84ea-2813ec4a119c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:39.948079 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-5d2918b0-8800-41d2-84ea-2813ec4a119c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 03:32:39.948079 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:39.948079 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:39.948079 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:39.948079 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:39.948079 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:39.948079 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:39.948079 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:39.948079 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:39.948079 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:39.948079 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:39.948079 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:39.948079 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:39.948079 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:39.948079 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:39.948079 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:39.949569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:39.949569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:39.949569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:39.949569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:39.949569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:39.949569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:39.949569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:39.949569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:39.949569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:39.949569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:39.949569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:39.949569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:39.949569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:39.949569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:39.949569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:39.949569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:39.950346 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:39.950346 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:39.950346 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:39.950346 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:39.950346 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:39.950346 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:39.950346 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:39.950346 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:39.950346 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:39.950346 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:39.950346 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 17 03:32:39.950346 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:39.951646 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 144/283] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 17 03:32:39.951856 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 139/283] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:32:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:39.959858 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da6ac2-3afd-4394-bb0d-84063c58ebe8 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:40.099243 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68da6ac2-3afd-4394-bb0d-84063c58ebe8 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:40.101619 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:40.133944 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd4dec-8908-42d3-be25-060545f4b2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:40.134559 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd4dec-8908-42d3-be25-060545f4b2e3 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:40.134846 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd4dec-8908-42d3-be25-060545f4b2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:40.135120 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bd4dec-8908-42d3-be25-060545f4b2e3 None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:40.135805 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd4dec-8908-42d3-be25-060545f4b2e3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:40.136110 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd4dec-8908-42d3-be25-060545f4b2e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:40.138105 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bd4dec-8908-42d3-be25-060545f4b2e3 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:40.152471 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 145/284] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Oct 17 03:32:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:40.158214 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2918b0-8800-41d2-84ea-2813ec4a119c None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:40.306779 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2918b0-8800-41d2-84ea-2813ec4a119c None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:40.344368 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd4dec-8908-42d3-be25-060545f4b2e3 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:40.351147 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41158bc9-1a13-4b3a-85d7-cbf5778b3297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:40.351815 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41158bc9-1a13-4b3a-85d7-cbf5778b3297 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:40.352098 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41158bc9-1a13-4b3a-85d7-cbf5778b3297 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:40.352368 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41158bc9-1a13-4b3a-85d7-cbf5778b3297 None admin] PATH_INFO: `/v3/users/9d0a16b904824ce1940e89d6554bc4b1` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:40.354585 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41158bc9-1a13-4b3a-85d7-cbf5778b3297 None admin] RBAC: Authorizing `identity:get_user(user_id=9d0a16b904824ce1940e89d6554bc4b1)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:40.356098 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41158bc9-1a13-4b3a-85d7-cbf5778b3297 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:40.360991 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 140/285] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:32:40 2022] GET /identity/v3/users/9d0a16b904824ce1940e89d6554bc4b1 => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:40.361843 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bd4dec-8908-42d3-be25-060545f4b2e3 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:40.364501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41158bc9-1a13-4b3a-85d7-cbf5778b3297 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:40.370566 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e95fd-bb6e-4381-a3db-bb78c6af9488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:40.371748 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e95fd-bb6e-4381-a3db-bb78c6af9488 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:40.372048 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e95fd-bb6e-4381-a3db-bb78c6af9488 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:40.372316 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e95fd-bb6e-4381-a3db-bb78c6af9488 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:40.398158 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e95fd-bb6e-4381-a3db-bb78c6af9488 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:40.405683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-9b0e95fd-bb6e-4381-a3db-bb78c6af9488 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:32:40.405683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:40.405683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:40.405683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:40.405683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:40.405683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:40.405683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:40.405683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:40.405683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:40.405683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:40.405683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:40.405683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:40.405683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:40.405683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:40.405683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:40.405683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:40.406528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:40.406528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:40.406528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:40.406528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:40.406528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:40.406528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:40.406528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:40.406528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:40.406528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:40.406528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:40.406528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:40.406528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:40.406528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:40.406528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:40.406528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:40.406528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:40.407290 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:40.407290 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:40.407290 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:40.407290 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:40.407290 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:40.407290 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:40.407290 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:40.407290 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:40.407290 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:40.407290 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:40.407290 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:32:40.407290 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:40.407290 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 146/286] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Oct 17 03:32:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 63 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:40.414283 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e95fd-bb6e-4381-a3db-bb78c6af9488 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:40.420090 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:40.436937 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e95fd-bb6e-4381-a3db-bb78c6af9488 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:40.445196 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed215089-d88b-4caa-bd39-6d625a1bad13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:40.445859 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed215089-d88b-4caa-bd39-6d625a1bad13 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:40.446142 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed215089-d88b-4caa-bd39-6d625a1bad13 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:40.446412 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed215089-d88b-4caa-bd39-6d625a1bad13 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:40.447186 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed215089-d88b-4caa-bd39-6d625a1bad13 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:40.447490 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed215089-d88b-4caa-bd39-6d625a1bad13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:40.453736 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed215089-d88b-4caa-bd39-6d625a1bad13 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:40.468489 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 147/287] 10.222.0.17 () {56 vars in 1120 bytes} [Mon Oct 17 03:32:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:40.471776 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed215089-d88b-4caa-bd39-6d625a1bad13 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:40.486860 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed215089-d88b-4caa-bd39-6d625a1bad13 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:40.496877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d6062-1a20-4e51-a6d0-1022405a248b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:40.497885 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d6062-1a20-4e51-a6d0-1022405a248b admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:40.498163 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d6062-1a20-4e51-a6d0-1022405a248b admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:40.498429 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d6062-1a20-4e51-a6d0-1022405a248b admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:40.522762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d6062-1a20-4e51-a6d0-1022405a248b admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:40.531018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-a65d6062-1a20-4e51-a6d0-1022405a248b admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 03:32:40.531018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:40.531018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:40.531018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:40.531018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:40.531018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:40.531018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:40.531018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:40.531018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:40.531018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:40.531018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:40.531018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:40.531018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:40.531018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:40.531018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:40.531018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:40.531877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:40.531877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:40.531877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:40.531877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:40.531877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:40.531877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:40.531877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:40.531877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:40.531877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:40.531877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:40.531877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:40.531877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:40.531877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:40.531877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:40.531877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:40.531877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:40.532643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:40.532643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:40.532643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:40.532643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:40.532643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:40.532643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:40.532643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:40.532643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:40.532643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:40.532643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:40.532643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 03:32:40.532643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:40.537440 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 148/288] 10.222.0.17 () {56 vars in 1135 bytes} [Mon Oct 17 03:32:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 68 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:40.540855 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d6062-1a20-4e51-a6d0-1022405a248b admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:40.557713 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d6062-1a20-4e51-a6d0-1022405a248b admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:40.561563 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41158bc9-1a13-4b3a-85d7-cbf5778b3297 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:40.570555 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce1123-3659-4a2c-9612-49f9f0ffb420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:40.572314 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce1123-3659-4a2c-9612-49f9f0ffb420 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:40.572607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce1123-3659-4a2c-9612-49f9f0ffb420 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:40.572879 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce1123-3659-4a2c-9612-49f9f0ffb420 admin admin] PATH_INFO: `/v3/groups/2f2aedb71aea48de9f2afd71a3301666` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:40.586030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce1123-3659-4a2c-9612-49f9f0ffb420 admin admin] RBAC: Authorizing `identity:get_group(group_id=2f2aedb71aea48de9f2afd71a3301666)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:40.590487 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce1123-3659-4a2c-9612-49f9f0ffb420 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:40.592150 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 149/289] 10.222.0.17 () {56 vars in 1201 bytes} [Mon Oct 17 03:32:40 2022] GET /identity/v3/groups/2f2aedb71aea48de9f2afd71a3301666 => generated 229 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:40.594111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce1123-3659-4a2c-9612-49f9f0ffb420 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:40.612730 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce1123-3659-4a2c-9612-49f9f0ffb420 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:40.624200 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885ef85-89e3-4027-88c5-35f933e295ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:40.624818 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885ef85-89e3-4027-88c5-35f933e295ba admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:40.625609 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885ef85-89e3-4027-88c5-35f933e295ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:40.625886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6885ef85-89e3-4027-88c5-35f933e295ba admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:40.627051 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6885ef85-89e3-4027-88c5-35f933e295ba admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:40.627357 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6885ef85-89e3-4027-88c5-35f933e295ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:40.630206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd789ffc-a634-46bb-8691-89ad48b3c15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:40.630870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd789ffc-a634-46bb-8691-89ad48b3c15b None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:40.632056 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6885ef85-89e3-4027-88c5-35f933e295ba admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:40.632704 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd789ffc-a634-46bb-8691-89ad48b3c15b None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:40.633015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd789ffc-a634-46bb-8691-89ad48b3c15b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:40.633798 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd789ffc-a634-46bb-8691-89ad48b3c15b None admin] RBAC: Adding query filter params (role.id=010a82aa9f3b4a23817fe234196cf259, user.id=9d0a16b904824ce1940e89d6554bc4b1, scope.system=all) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:40.634104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd789ffc-a634-46bb-8691-89ad48b3c15b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:40.637705 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd789ffc-a634-46bb-8691-89ad48b3c15b None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:40.650875 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 150/290] 10.222.0.17 () {56 vars in 1132 bytes} [Mon Oct 17 03:32:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:40.654153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885ef85-89e3-4027-88c5-35f933e295ba admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:40.662195 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 141/291] 10.222.0.17 () {56 vars in 1308 bytes} [Mon Oct 17 03:32:40 2022] GET /identity/v3/role_assignments?user.id=9d0a16b904824ce1940e89d6554bc4b1&scope.system=all&role.id=010a82aa9f3b4a23817fe234196cf259 => generated 510 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:40.664460 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd789ffc-a634-46bb-8691-89ad48b3c15b None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:40.674349 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6885ef85-89e3-4027-88c5-35f933e295ba admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:40.688408 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49857785-0c26-4681-8410-2f1b6cbd92db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:40.689972 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49857785-0c26-4681-8410-2f1b6cbd92db admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:40.690251 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49857785-0c26-4681-8410-2f1b6cbd92db admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:40.690526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49857785-0c26-4681-8410-2f1b6cbd92db admin admin] PATH_INFO: `/v3/projects/6f518d7d57a2406a8345926faf8a2f14` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:40.693235 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd789ffc-a634-46bb-8691-89ad48b3c15b None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:40.694633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49857785-0c26-4681-8410-2f1b6cbd92db admin admin] RBAC: Authorizing `identity:get_project(project_id=6f518d7d57a2406a8345926faf8a2f14)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:40.703509 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49857785-0c26-4681-8410-2f1b6cbd92db admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:40.708380 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a89bf0-7637-44e6-8d1a-b6672a130d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:40.711582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 151/292] 10.222.0.17 () {56 vars in 1207 bytes} [Mon Oct 17 03:32:40 2022] GET /identity/v3/projects/6f518d7d57a2406a8345926faf8a2f14 => generated 344 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:40.711801 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a89bf0-7637-44e6-8d1a-b6672a130d25 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:40.711801 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a89bf0-7637-44e6-8d1a-b6672a130d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:40.711801 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a89bf0-7637-44e6-8d1a-b6672a130d25 admin admin] PATH_INFO: `/v3/groups/32be849bef724c94a2e14a0e8b62b35d` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:40.712746 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a89bf0-7637-44e6-8d1a-b6672a130d25 admin admin] RBAC: Authorizing `identity:get_group(group_id=32be849bef724c94a2e14a0e8b62b35d)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:40.715484 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a89bf0-7637-44e6-8d1a-b6672a130d25 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:40.717289 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49857785-0c26-4681-8410-2f1b6cbd92db admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:40.719617 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 142/293] 10.222.0.17 () {56 vars in 1201 bytes} [Mon Oct 17 03:32:40 2022] GET /identity/v3/groups/32be849bef724c94a2e14a0e8b62b35d => generated 226 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:40.726722 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a89bf0-7637-44e6-8d1a-b6672a130d25 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:40.732751 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49857785-0c26-4681-8410-2f1b6cbd92db admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:40.737633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c70d94-5739-4df0-a2b8-fbef5034b11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:40.738451 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c70d94-5739-4df0-a2b8-fbef5034b11f admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:40.738825 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c70d94-5739-4df0-a2b8-fbef5034b11f admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:40.739235 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c70d94-5739-4df0-a2b8-fbef5034b11f admin admin] PATH_INFO: `/v3/projects/6f518d7d57a2406a8345926faf8a2f14/groups/2f2aedb71aea48de9f2afd71a3301666/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:40.741384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c70d94-5739-4df0-a2b8-fbef5034b11f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f518d7d57a2406a8345926faf8a2f14, group_id=2f2aedb71aea48de9f2afd71a3301666, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:40.744874 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c70d94-5739-4df0-a2b8-fbef5034b11f admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:40.745174 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a89bf0-7637-44e6-8d1a-b6672a130d25 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:40.761859 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788edbb-f224-4e2f-a4b3-e35834e33666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:40.762508 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788edbb-f224-4e2f-a4b3-e35834e33666 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:40.764455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788edbb-f224-4e2f-a4b3-e35834e33666 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:40.764455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788edbb-f224-4e2f-a4b3-e35834e33666 admin admin] PATH_INFO: `/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:40.764455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788edbb-f224-4e2f-a4b3-e35834e33666 admin admin] RBAC: Authorizing `identity:get_project(project_id=9a1236f1bb0f4a499c0206128e5c97f9)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:40.766795 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788edbb-f224-4e2f-a4b3-e35834e33666 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:40.768516 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 143/294] 10.222.0.17 () {56 vars in 1207 bytes} [Mon Oct 17 03:32:40 2022] GET /identity/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9 => generated 302 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:40.774505 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788edbb-f224-4e2f-a4b3-e35834e33666 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:40.789691 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788edbb-f224-4e2f-a4b3-e35834e33666 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:40.795272 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94d89c-9621-4f6c-b094-bbfe2483fadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:40.796144 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94d89c-9621-4f6c-b094-bbfe2483fadd admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:40.796431 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94d89c-9621-4f6c-b094-bbfe2483fadd admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:40.796701 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94d89c-9621-4f6c-b094-bbfe2483fadd admin admin] PATH_INFO: `/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9/groups/32be849bef724c94a2e14a0e8b62b35d/roles/609a4bf154ff4dcabb408b3789d28983` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:40.799667 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c94d89c-9621-4f6c-b094-bbfe2483fadd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a1236f1bb0f4a499c0206128e5c97f9, group_id=32be849bef724c94a2e14a0e8b62b35d, role_id=609a4bf154ff4dcabb408b3789d28983)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:40.802615 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c94d89c-9621-4f6c-b094-bbfe2483fadd admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:40.855860 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 152/295] 10.222.0.17 () {58 vars in 1463 bytes} [Mon Oct 17 03:32:40 2022] PUT /identity/v3/projects/6f518d7d57a2406a8345926faf8a2f14/groups/2f2aedb71aea48de9f2afd71a3301666/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:32:40.877740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 144/296] 10.222.0.17 () {58 vars in 1463 bytes} [Mon Oct 17 03:32:40 2022] PUT /identity/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9/groups/32be849bef724c94a2e14a0e8b62b35d/roles/609a4bf154ff4dcabb408b3789d28983 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:32:41.421053 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074291fa-97c1-4bc5-a6c7-c7e7697e8381 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:41.421422 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074291fa-97c1-4bc5-a6c7-c7e7697e8381 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:41.421600 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074291fa-97c1-4bc5-a6c7-c7e7697e8381 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:41.423217 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 153/297] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:41.428443 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51806e8-d77d-4806-877c-582039743203 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:41.428749 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51806e8-d77d-4806-877c-582039743203 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:41.429002 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51806e8-d77d-4806-877c-582039743203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:41.452786 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-f51806e8-d77d-4806-877c-582039743203 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:41.454009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-f51806e8-d77d-4806-877c-582039743203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:41.530210 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:41.542309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 145/298] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:41.755863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72302cd-6afc-43a0-ae71-b4b06216c014 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:41.756390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72302cd-6afc-43a0-ae71-b4b06216c014 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:41.756390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72302cd-6afc-43a0-ae71-b4b06216c014 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:41.757950 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 154/299] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:41.763435 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a797dad-440a-4a96-8190-acb8cafe7c58 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:41.763854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a797dad-440a-4a96-8190-acb8cafe7c58 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:41.764020 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a797dad-440a-4a96-8190-acb8cafe7c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:41.789146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-5a797dad-440a-4a96-8190-acb8cafe7c58 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:41.790335 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-5a797dad-440a-4a96-8190-acb8cafe7c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:41.880066 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 146/300] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:41.999195 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72302cd-6afc-43a0-ae71-b4b06216c014 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:42.124377 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:42.128826 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72302cd-6afc-43a0-ae71-b4b06216c014 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:42.167026 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d02e36-3c2c-484a-98fc-13772cd975b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:42.168295 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d02e36-3c2c-484a-98fc-13772cd975b0 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:42.168909 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d02e36-3c2c-484a-98fc-13772cd975b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:42.169537 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d02e36-3c2c-484a-98fc-13772cd975b0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:42.180730 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d02e36-3c2c-484a-98fc-13772cd975b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:42.185726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-66d02e36-3c2c-484a-98fc-13772cd975b0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 03:32:42.185726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:42.185726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:42.185726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:42.185726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:42.185726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:42.185726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:42.185726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:42.185726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:42.185726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:42.185726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:42.185726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:42.185726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:42.185726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:42.185726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:42.185726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:42.186582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:42.186582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:42.186582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:42.186582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:42.186582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:42.186582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:42.186582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:42.186582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:42.186582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:42.186582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:42.186582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:42.186582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:42.186582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:42.186582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:42.186582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:42.186582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:42.187335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:42.187335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:42.187335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:42.187335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:42.187335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:42.187335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:42.187335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:42.187335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:42.187335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:42.187335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:42.187335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 03:32:42.187335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:42.187335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 155/301] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Oct 17 03:32:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:42.193415 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a797dad-440a-4a96-8190-acb8cafe7c58 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:42.266390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d02e36-3c2c-484a-98fc-13772cd975b0 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:42.333732 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a797dad-440a-4a96-8190-acb8cafe7c58 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:42.382458 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:42.404952 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a98c5-1904-46fc-8c16-abccedbee4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:42.405580 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a98c5-1904-46fc-8c16-abccedbee4e2 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:42.405863 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a98c5-1904-46fc-8c16-abccedbee4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:42.406134 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95a98c5-1904-46fc-8c16-abccedbee4e2 None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:42.406834 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95a98c5-1904-46fc-8c16-abccedbee4e2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:42.407165 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95a98c5-1904-46fc-8c16-abccedbee4e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:42.408560 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95a98c5-1904-46fc-8c16-abccedbee4e2 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:42.425623 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 147/302] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:32:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:42.431998 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a98c5-1904-46fc-8c16-abccedbee4e2 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:42.466530 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d02e36-3c2c-484a-98fc-13772cd975b0 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:42.527825 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec40da1-3b4c-4ffa-b3d9-33ca6e224a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:42.528397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec40da1-3b4c-4ffa-b3d9-33ca6e224a8c None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:42.528680 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec40da1-3b4c-4ffa-b3d9-33ca6e224a8c None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:42.528954 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec40da1-3b4c-4ffa-b3d9-33ca6e224a8c None admin] PATH_INFO: `/v3/roles/member` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:42.540170 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec40da1-3b4c-4ffa-b3d9-33ca6e224a8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:42.546058 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-bec40da1-3b4c-4ffa-b3d9-33ca6e224a8c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 03:32:42.546058 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:42.546058 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:42.546058 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:42.546058 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:42.546058 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:42.546058 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:42.546058 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:42.546058 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:42.546058 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:42.546058 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:42.546058 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:42.546058 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:42.546058 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:42.546058 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:42.546058 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:42.546945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:42.546945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:42.546945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:42.546945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:42.546945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:42.546945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:42.546945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:42.546945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:42.546945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:42.546945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:42.546945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:42.546945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:42.546945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:42.546945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:42.546945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:42.546945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:42.547762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:42.547762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:42.547762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:42.547762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:42.547762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:42.547762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:42.547762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:42.547762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:42.547762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:42.547762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:42.547762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 03:32:42.547762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:42.547762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 156/303] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:32:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:32:42.548413 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5df92-cb70-482a-89c0-806c58024356 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:42.548413 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5df92-cb70-482a-89c0-806c58024356 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:42.548413 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d5df92-cb70-482a-89c0-806c58024356 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:42.549581 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 157/304] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:42.555402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677b9ba-b3c0-4e19-8861-ff44a9eba305 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:42.555402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677b9ba-b3c0-4e19-8861-ff44a9eba305 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:42.555596 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677b9ba-b3c0-4e19-8861-ff44a9eba305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:42.579517 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-c677b9ba-b3c0-4e19-8861-ff44a9eba305 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:42.580754 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c677b9ba-b3c0-4e19-8861-ff44a9eba305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:42.588533 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95a98c5-1904-46fc-8c16-abccedbee4e2 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:42.596985 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:42.639277 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2ac20-8291-4a9a-8233-f9711a895fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:42.640446 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2ac20-8291-4a9a-8233-f9711a895fc0 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:42.640734 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2ac20-8291-4a9a-8233-f9711a895fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:42.641404 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c2ac20-8291-4a9a-8233-f9711a895fc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:42.669403 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 158/305] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:42.670325 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677b9ba-b3c0-4e19-8861-ff44a9eba305 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:42.708642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:42.723636 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c2ac20-8291-4a9a-8233-f9711a895fc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:42.725409 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c2ac20-8291-4a9a-8233-f9711a895fc0 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:42.835659 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 148/306] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:32:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 17 03:32:42.841311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2ac20-8291-4a9a-8233-f9711a895fc0 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:42.860124 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677b9ba-b3c0-4e19-8861-ff44a9eba305 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:42.929348 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33eba9-fc85-4645-b2cb-4d2e7ce9bbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:42.930206 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33eba9-fc85-4645-b2cb-4d2e7ce9bbcb None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:42.930631 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33eba9-fc85-4645-b2cb-4d2e7ce9bbcb None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:42.931070 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee33eba9-fc85-4645-b2cb-4d2e7ce9bbcb None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:42.931901 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee33eba9-fc85-4645-b2cb-4d2e7ce9bbcb None admin] RBAC: Adding query filter params (name=member) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:42.934412 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee33eba9-fc85-4645-b2cb-4d2e7ce9bbcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:42.935968 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee33eba9-fc85-4645-b2cb-4d2e7ce9bbcb None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:42.952277 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 159/307] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Oct 17 03:32:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:42.966584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33eba9-fc85-4645-b2cb-4d2e7ce9bbcb None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:42.989323 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:43.018160 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:43.030815 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c2ac20-8291-4a9a-8233-f9711a895fc0 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:43.094814 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b019e74-3323-4ed9-9a4c-7e3856274bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:43.095455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b019e74-3323-4ed9-9a4c-7e3856274bf0 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:43.095851 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b019e74-3323-4ed9-9a4c-7e3856274bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:43.096196 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b019e74-3323-4ed9-9a4c-7e3856274bf0 None admin] PATH_INFO: `/v3/projects/bc8197db3d20408990645225920f63d7` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:43.098195 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b019e74-3323-4ed9-9a4c-7e3856274bf0 None admin] RBAC: Authorizing `identity:get_project(project_id=bc8197db3d20408990645225920f63d7)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:43.100377 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b019e74-3323-4ed9-9a4c-7e3856274bf0 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:43.102604 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 149/308] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:32:42 2022] GET /identity/v3/projects/bc8197db3d20408990645225920f63d7 => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:43.104091 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b019e74-3323-4ed9-9a4c-7e3856274bf0 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:43.110197 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee33eba9-fc85-4645-b2cb-4d2e7ce9bbcb None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:43.151223 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85584f53-4ac7-4f01-b81c-f00c40478f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:43.152534 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85584f53-4ac7-4f01-b81c-f00c40478f51 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:43.152885 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85584f53-4ac7-4f01-b81c-f00c40478f51 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:43.153225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85584f53-4ac7-4f01-b81c-f00c40478f51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:43.235472 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b019e74-3323-4ed9-9a4c-7e3856274bf0 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:43.238898 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85584f53-4ac7-4f01-b81c-f00c40478f51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:43.240474 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85584f53-4ac7-4f01-b81c-f00c40478f51 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:43.254059 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:43.271415 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960208a-5920-4a83-8ff2-3a80a90bd887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:43.272074 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960208a-5920-4a83-8ff2-3a80a90bd887 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:43.272360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960208a-5920-4a83-8ff2-3a80a90bd887 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:43.272692 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960208a-5920-4a83-8ff2-3a80a90bd887 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:43.283873 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a960208a-5920-4a83-8ff2-3a80a90bd887 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:43.289363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-a960208a-5920-4a83-8ff2-3a80a90bd887 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 17 03:32:43.289363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:43.289363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:43.289363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:43.289363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:43.289363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:43.289363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:43.289363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:43.289363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:43.289363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:43.289363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:43.289363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:43.289363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:43.289363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:43.289363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:43.289363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:43.290202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:43.290202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:43.290202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:43.290202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:43.290202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:43.290202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:43.290202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:43.290202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:43.290202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:43.290202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:43.290202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:43.290202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:43.290202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:43.290202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:43.290202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:43.290202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:43.292637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:43.292637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:43.292637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:43.292637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:43.292637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:43.292637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:43.292637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:43.292637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:43.292637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:43.292637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:43.292637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 17 03:32:43.292637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:43.292637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 150/309] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:32:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:43.292637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960208a-5920-4a83-8ff2-3a80a90bd887 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:43.315585 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:43.358113 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 160/310] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:32:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 17 03:32:43.359287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85584f53-4ac7-4f01-b81c-f00c40478f51 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:43.423323 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960208a-5920-4a83-8ff2-3a80a90bd887 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:43.469778 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bddc35-149c-4cfc-8b24-11dc027b36fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:43.470667 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bddc35-149c-4cfc-8b24-11dc027b36fc None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:43.471116 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bddc35-149c-4cfc-8b24-11dc027b36fc None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:43.471530 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bddc35-149c-4cfc-8b24-11dc027b36fc None admin] PATH_INFO: `/v3/groups/32be849bef724c94a2e14a0e8b62b35d` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:43.471749 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85584f53-4ac7-4f01-b81c-f00c40478f51 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:43.475090 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bddc35-149c-4cfc-8b24-11dc027b36fc None admin] RBAC: Authorizing `identity:get_group(group_id=32be849bef724c94a2e14a0e8b62b35d)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:43.477299 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bddc35-149c-4cfc-8b24-11dc027b36fc None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:43.478273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 151/311] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Oct 17 03:32:43 2022] GET /identity/v3/groups/32be849bef724c94a2e14a0e8b62b35d => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:43.479049 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bddc35-149c-4cfc-8b24-11dc027b36fc None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:43.487529 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:43.513999 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa4f4e-cd98-4d73-adea-1dd61dcdfcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:43.514911 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa4f4e-cd98-4d73-adea-1dd61dcdfcfa None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:43.515393 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa4f4e-cd98-4d73-adea-1dd61dcdfcfa None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:43.515863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa4f4e-cd98-4d73-adea-1dd61dcdfcfa None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:43.516747 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fa4f4e-cd98-4d73-adea-1dd61dcdfcfa None admin] RBAC: Adding query filter params (name=member) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:43.517218 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fa4f4e-cd98-4d73-adea-1dd61dcdfcfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:43.518809 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fa4f4e-cd98-4d73-adea-1dd61dcdfcfa None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:43.529085 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 161/312] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Oct 17 03:32:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:43.534778 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa4f4e-cd98-4d73-adea-1dd61dcdfcfa None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:43.573335 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:43.594082 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bddc35-149c-4cfc-8b24-11dc027b36fc None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:43.631629 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8543d0-b922-4c53-9cad-e57ac2a0d6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:43.632547 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8543d0-b922-4c53-9cad-e57ac2a0d6c0 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:43.633045 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8543d0-b922-4c53-9cad-e57ac2a0d6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:43.633512 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8543d0-b922-4c53-9cad-e57ac2a0d6c0 None admin] PATH_INFO: `/v3/projects/bc8197db3d20408990645225920f63d7` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:43.635275 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8543d0-b922-4c53-9cad-e57ac2a0d6c0 None admin] RBAC: Authorizing `identity:get_project(project_id=bc8197db3d20408990645225920f63d7)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:43.638843 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8543d0-b922-4c53-9cad-e57ac2a0d6c0 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:43.641338 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 152/313] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:32:43 2022] GET /identity/v3/projects/bc8197db3d20408990645225920f63d7 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:43.644126 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8543d0-b922-4c53-9cad-e57ac2a0d6c0 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:43.654795 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fa4f4e-cd98-4d73-adea-1dd61dcdfcfa None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:43.707280 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc0148-4c08-4aec-9251-9acc28d710b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:43.708274 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc0148-4c08-4aec-9251-9acc28d710b6 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:43.708769 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc0148-4c08-4aec-9251-9acc28d710b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:43.709196 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc0148-4c08-4aec-9251-9acc28d710b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:43.710095 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc0148-4c08-4aec-9251-9acc28d710b6 None admin] RBAC: Adding query filter params (group.id=32be849bef724c94a2e14a0e8b62b35d, role.id=609a4bf154ff4dcabb408b3789d28983, scope.project.id=bc8197db3d20408990645225920f63d7) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:43.710686 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc0148-4c08-4aec-9251-9acc28d710b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:43.712500 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc0148-4c08-4aec-9251-9acc28d710b6 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:43.759294 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 162/314] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Oct 17 03:32:43 2022] GET /identity/v3/role_assignments?group.id=32be849bef724c94a2e14a0e8b62b35d&scope.project.id=bc8197db3d20408990645225920f63d7&role.id=609a4bf154ff4dcabb408b3789d28983 => generated 611 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:43.760619 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc0148-4c08-4aec-9251-9acc28d710b6 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:43.792175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8543d0-b922-4c53-9cad-e57ac2a0d6c0 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:43.833791 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:43.835004 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9851d6-7012-456f-bc5b-3970c8af191b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:43.835649 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9851d6-7012-456f-bc5b-3970c8af191b None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:43.835963 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9851d6-7012-456f-bc5b-3970c8af191b None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:43.836489 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9851d6-7012-456f-bc5b-3970c8af191b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:43.862526 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:43.907007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc0148-4c08-4aec-9251-9acc28d710b6 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:43.926679 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9851d6-7012-456f-bc5b-3970c8af191b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:43.926679 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9851d6-7012-456f-bc5b-3970c8af191b None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:43.948148 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee268b3c-45ac-4bdf-84fd-9bfb92548acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:43.950129 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee268b3c-45ac-4bdf-84fd-9bfb92548acb None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:43.950623 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee268b3c-45ac-4bdf-84fd-9bfb92548acb None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:43.951126 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee268b3c-45ac-4bdf-84fd-9bfb92548acb None admin] PATH_INFO: `/v3/groups/32be849bef724c94a2e14a0e8b62b35d` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:43.953422 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee268b3c-45ac-4bdf-84fd-9bfb92548acb None admin] RBAC: Authorizing `identity:get_group(group_id=32be849bef724c94a2e14a0e8b62b35d)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:43.954934 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee268b3c-45ac-4bdf-84fd-9bfb92548acb None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:43.957688 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 163/315] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Oct 17 03:32:43 2022] GET /identity/v3/groups/32be849bef724c94a2e14a0e8b62b35d => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:43.959259 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca004a1-d1c6-48ad-83ab-6f4dabb126df None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:43.960088 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca004a1-d1c6-48ad-83ab-6f4dabb126df None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:43.960374 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca004a1-d1c6-48ad-83ab-6f4dabb126df None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:43.962657 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 164/316] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:43.963995 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538db45d-31df-424b-862a-82a302ba8874 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:43.964322 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538db45d-31df-424b-862a-82a302ba8874 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:43.964657 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538db45d-31df-424b-862a-82a302ba8874 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:43.966442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 165/317] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:43.967041 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538db45d-31df-424b-862a-82a302ba8874 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:44.060001 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 153/318] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:32:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 03:32:44.061937 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e223bd-8f4d-43ad-ae34-4d400b1ced09 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:44.062227 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e223bd-8f4d-43ad-ae34-4d400b1ced09 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:44.062497 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e223bd-8f4d-43ad-ae34-4d400b1ced09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:44.082762 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-96e223bd-8f4d-43ad-ae34-4d400b1ced09 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:44.083991 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-96e223bd-8f4d-43ad-ae34-4d400b1ced09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:44.100768 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538db45d-31df-424b-862a-82a302ba8874 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:44.104270 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:44.139916 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:44.144537 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94315fa-90f2-412a-83bd-4f2b33a5dfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:44.145824 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94315fa-90f2-412a-83bd-4f2b33a5dfc5 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:44.145824 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94315fa-90f2-412a-83bd-4f2b33a5dfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:44.146064 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94315fa-90f2-412a-83bd-4f2b33a5dfc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:44.147000 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94315fa-90f2-412a-83bd-4f2b33a5dfc5 None admin] RBAC: Adding query filter params (group.id=32be849bef724c94a2e14a0e8b62b35d, role.id=84a147decc7b417783086c1effe0fe20, scope.project.id=bc8197db3d20408990645225920f63d7) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:44.147311 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94315fa-90f2-412a-83bd-4f2b33a5dfc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:44.148802 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94315fa-90f2-412a-83bd-4f2b33a5dfc5 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:44.157225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 166/319] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Oct 17 03:32:43 2022] GET /identity/v3/role_assignments?group.id=32be849bef724c94a2e14a0e8b62b35d&scope.project.id=bc8197db3d20408990645225920f63d7&role.id=84a147decc7b417783086c1effe0fe20 => generated 611 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:44.158341 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e36e5-bc10-4046-a8b1-b2602c34607a None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:44.158667 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e36e5-bc10-4046-a8b1-b2602c34607a None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:44.159400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e36e5-bc10-4046-a8b1-b2602c34607a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:44.170977 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 154/320] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:44.176909 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e223bd-8f4d-43ad-ae34-4d400b1ced09 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:44.184125 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-537e36e5-bc10-4046-a8b1-b2602c34607a None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:44.185358 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-537e36e5-bc10-4046-a8b1-b2602c34607a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:44.276846 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 167/321] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:44.306331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e223bd-8f4d-43ad-ae34-4d400b1ced09 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:44.348894 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b5830c-b809-4fbe-b01d-0db313b877b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:44.349541 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5830c-b809-4fbe-b01d-0db313b877b0 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:44.349825 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5830c-b809-4fbe-b01d-0db313b877b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:44.350103 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5830c-b809-4fbe-b01d-0db313b877b0 None admin] PATH_INFO: `/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:44.351232 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b5830c-b809-4fbe-b01d-0db313b877b0 None admin] RBAC: Authorizing `identity:get_project(project_id=9a1236f1bb0f4a499c0206128e5c97f9)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:44.352823 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b5830c-b809-4fbe-b01d-0db313b877b0 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:44.354482 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 155/322] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:32:44 2022] GET /identity/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:44.360791 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e36e5-bc10-4046-a8b1-b2602c34607a None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:44.462336 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:44.466957 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e36e5-bc10-4046-a8b1-b2602c34607a None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:44.491224 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b5830c-b809-4fbe-b01d-0db313b877b0 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:44.506700 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d80ce-ccc0-4e84-b1bb-60cc46036708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:44.507676 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d80ce-ccc0-4e84-b1bb-60cc46036708 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:44.508205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d80ce-ccc0-4e84-b1bb-60cc46036708 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:44.508648 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d80ce-ccc0-4e84-b1bb-60cc46036708 None admin] PATH_INFO: `/v3/groups/32be849bef724c94a2e14a0e8b62b35d` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:44.511080 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d80ce-ccc0-4e84-b1bb-60cc46036708 None admin] RBAC: Authorizing `identity:get_group(group_id=32be849bef724c94a2e14a0e8b62b35d)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:44.512703 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d80ce-ccc0-4e84-b1bb-60cc46036708 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:44.514921 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 168/323] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Oct 17 03:32:44 2022] GET /identity/v3/groups/32be849bef724c94a2e14a0e8b62b35d => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:44.520528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d80ce-ccc0-4e84-b1bb-60cc46036708 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:44.565809 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:44.633829 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b5830c-b809-4fbe-b01d-0db313b877b0 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:44.647933 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d80ce-ccc0-4e84-b1bb-60cc46036708 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:44.676433 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71814c4-bd72-4276-93bb-294083118ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:44.676733 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71814c4-bd72-4276-93bb-294083118ce2 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:44.676998 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71814c4-bd72-4276-93bb-294083118ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:44.677275 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71814c4-bd72-4276-93bb-294083118ce2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:44.688252 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86bfb27-39c8-4299-afd7-d0af32dbb42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:44.689605 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bfb27-39c8-4299-afd7-d0af32dbb42f None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:44.689605 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bfb27-39c8-4299-afd7-d0af32dbb42f None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:44.689605 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86bfb27-39c8-4299-afd7-d0af32dbb42f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:44.690143 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86bfb27-39c8-4299-afd7-d0af32dbb42f None admin] RBAC: Adding query filter params (group.id=32be849bef724c94a2e14a0e8b62b35d, role.id=84a147decc7b417783086c1effe0fe20, scope.project.id=9a1236f1bb0f4a499c0206128e5c97f9) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:44.690452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86bfb27-39c8-4299-afd7-d0af32dbb42f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:44.691922 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86bfb27-39c8-4299-afd7-d0af32dbb42f None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:44.693088 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71814c4-bd72-4276-93bb-294083118ce2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:44.699342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-a71814c4-bd72-4276-93bb-294083118ce2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 03:32:44.699342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:44.699342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:44.699342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:44.699342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:44.699342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:44.699342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:44.699342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:44.699342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:44.699342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:44.699342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:44.699342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:44.699342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:44.699342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:44.699342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:44.699342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:44.700371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:44.700371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:44.700371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:44.700371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:44.700371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:44.700371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:44.700371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:44.700371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:44.700371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:44.700371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:44.700371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:44.700371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:44.700371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:44.700371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:44.700371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:44.700371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:44.701128 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:44.701128 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:44.701128 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:44.701128 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:44.701128 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:44.701128 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:44.701128 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:44.701128 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:44.701128 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:44.701128 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:44.701128 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 03:32:44.701128 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:44.701128 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 156/324] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Oct 17 03:32:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:44.701128 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71814c4-bd72-4276-93bb-294083118ce2 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:44.703899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 169/325] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Oct 17 03:32:44 2022] GET /identity/v3/role_assignments?group.id=32be849bef724c94a2e14a0e8b62b35d&scope.project.id=9a1236f1bb0f4a499c0206128e5c97f9&role.id=84a147decc7b417783086c1effe0fe20 => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:44.709100 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86bfb27-39c8-4299-afd7-d0af32dbb42f None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:44.746065 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:44.837330 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71814c4-bd72-4276-93bb-294083118ce2 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:44.846750 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86bfb27-39c8-4299-afd7-d0af32dbb42f None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:44.850786 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:44.885479 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a8494-8526-4997-9b33-74d5d7bcec80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:44.886160 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a8494-8526-4997-9b33-74d5d7bcec80 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:44.886482 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a8494-8526-4997-9b33-74d5d7bcec80 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:44.887866 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a8494-8526-4997-9b33-74d5d7bcec80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:44.892537 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262742b3-f68c-42fe-9f99-7277ed82b710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:44.893305 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262742b3-f68c-42fe-9f99-7277ed82b710 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:44.893735 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262742b3-f68c-42fe-9f99-7277ed82b710 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:44.894018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262742b3-f68c-42fe-9f99-7277ed82b710 None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:44.894690 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262742b3-f68c-42fe-9f99-7277ed82b710 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:44.894995 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262742b3-f68c-42fe-9f99-7277ed82b710 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:44.896910 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262742b3-f68c-42fe-9f99-7277ed82b710 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:44.899273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a8494-8526-4997-9b33-74d5d7bcec80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:44.905175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-8f9a8494-8526-4997-9b33-74d5d7bcec80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:32:44.905175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:44.905175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:44.905175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:44.905175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:44.905175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:44.905175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:44.905175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:44.905175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:44.905175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:44.905175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:44.905175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:44.905175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:44.905175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:44.905175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:44.905175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:44.906269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:44.906269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:44.906269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:44.906269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:44.906269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:44.906269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:44.906269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:44.906269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:44.906269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:44.906269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:44.906269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:44.906269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:44.906269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:44.906269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:44.906269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:44.906269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:44.907254 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:44.907254 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:44.907254 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:44.907254 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:44.907254 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:44.907254 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:44.907254 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:44.907254 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:44.907254 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:44.907254 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:44.907254 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:32:44.907254 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:44.908071 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 157/326] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Oct 17 03:32:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:44.910442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 170/327] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:32:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:44.912209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a8494-8526-4997-9b33-74d5d7bcec80 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:44.916777 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262742b3-f68c-42fe-9f99-7277ed82b710 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:45.027580 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a8494-8526-4997-9b33-74d5d7bcec80 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:45.036657 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:45.044817 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262742b3-f68c-42fe-9f99-7277ed82b710 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:45.064296 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3d7e2-b78f-444f-aa87-42cc92d84a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:45.064911 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3d7e2-b78f-444f-aa87-42cc92d84a7b None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:45.065192 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3d7e2-b78f-444f-aa87-42cc92d84a7b None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:45.065466 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3d7e2-b78f-444f-aa87-42cc92d84a7b None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:45.066126 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f3d7e2-b78f-444f-aa87-42cc92d84a7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:45.066425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f3d7e2-b78f-444f-aa87-42cc92d84a7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:45.068417 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f3d7e2-b78f-444f-aa87-42cc92d84a7b None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:45.080351 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 158/328] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:32:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:45.085967 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3d7e2-b78f-444f-aa87-42cc92d84a7b None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:45.086402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e94697-1424-4133-b863-17cfee322f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:45.087138 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e94697-1424-4133-b863-17cfee322f43 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:45.087271 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e94697-1424-4133-b863-17cfee322f43 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:45.087544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e94697-1424-4133-b863-17cfee322f43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:45.122282 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:45.166599 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e94697-1424-4133-b863-17cfee322f43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:45.168127 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e94697-1424-4133-b863-17cfee322f43 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:45.202790 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3d7e2-b78f-444f-aa87-42cc92d84a7b None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:45.243569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ad9ab-d84b-43b3-a2a9-c706d56ac5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:45.243569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ad9ab-d84b-43b3-a2a9-c706d56ac5e8 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:45.243569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ad9ab-d84b-43b3-a2a9-c706d56ac5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:45.243569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ad9ab-d84b-43b3-a2a9-c706d56ac5e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:45.268475 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:45.284005 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 171/329] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:32:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 03:32:45.288626 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e94697-1424-4133-b863-17cfee322f43 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:45.321896 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ad9ab-d84b-43b3-a2a9-c706d56ac5e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:45.324297 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ad9ab-d84b-43b3-a2a9-c706d56ac5e8 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:45.358048 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:45.402822 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e94697-1424-4133-b863-17cfee322f43 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:45.438345 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 159/330] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:32:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 17 03:32:45.440175 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d966c5-5aea-4c76-9533-01103f2da654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:45.440820 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d966c5-5aea-4c76-9533-01103f2da654 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:45.441106 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d966c5-5aea-4c76-9533-01103f2da654 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:45.441376 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d966c5-5aea-4c76-9533-01103f2da654 None admin] PATH_INFO: `/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:45.442517 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d966c5-5aea-4c76-9533-01103f2da654 None admin] RBAC: Authorizing `identity:get_project(project_id=9a1236f1bb0f4a499c0206128e5c97f9)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:45.443953 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d966c5-5aea-4c76-9533-01103f2da654 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:45.444089 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ad9ab-d84b-43b3-a2a9-c706d56ac5e8 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:45.445706 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 172/331] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:32:45 2022] GET /identity/v3/projects/9a1236f1bb0f4a499c0206128e5c97f9 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:45.451645 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d966c5-5aea-4c76-9533-01103f2da654 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:45.508791 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:45.544505 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ad9ab-d84b-43b3-a2a9-c706d56ac5e8 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:45.549194 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d966c5-5aea-4c76-9533-01103f2da654 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:45.571858 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:45.577703 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073105cd-90ae-47cc-9a10-ec16b2672ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:45.580311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073105cd-90ae-47cc-9a10-ec16b2672ae0 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:45.580311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073105cd-90ae-47cc-9a10-ec16b2672ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:45.580311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073105cd-90ae-47cc-9a10-ec16b2672ae0 None admin] PATH_INFO: `/v3/projects/6f518d7d57a2406a8345926faf8a2f14` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:45.580311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073105cd-90ae-47cc-9a10-ec16b2672ae0 None admin] RBAC: Authorizing `identity:get_project(project_id=6f518d7d57a2406a8345926faf8a2f14)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:45.581454 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073105cd-90ae-47cc-9a10-ec16b2672ae0 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:45.582026 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecee9e5-958f-4667-a398-2b0c667498b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:45.582657 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecee9e5-958f-4667-a398-2b0c667498b7 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:45.582947 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecee9e5-958f-4667-a398-2b0c667498b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:45.583068 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 160/332] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:32:45 2022] GET /identity/v3/projects/6f518d7d57a2406a8345926faf8a2f14 => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:45.583222 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecee9e5-958f-4667-a398-2b0c667498b7 None admin] PATH_INFO: `/v3/groups/32be849bef724c94a2e14a0e8b62b35d` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:45.584837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecee9e5-958f-4667-a398-2b0c667498b7 None admin] RBAC: Authorizing `identity:get_group(group_id=32be849bef724c94a2e14a0e8b62b35d)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:45.585770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecee9e5-958f-4667-a398-2b0c667498b7 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:45.592198 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 173/333] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Oct 17 03:32:45 2022] GET /identity/v3/groups/32be849bef724c94a2e14a0e8b62b35d => generated 226 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:45.592323 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073105cd-90ae-47cc-9a10-ec16b2672ae0 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:45.593140 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecee9e5-958f-4667-a398-2b0c667498b7 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:45.690626 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073105cd-90ae-47cc-9a10-ec16b2672ae0 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:45.691775 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecee9e5-958f-4667-a398-2b0c667498b7 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:45.720333 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:45.724179 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0948145-1164-42b9-9da5-60846c422871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:45.724818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0948145-1164-42b9-9da5-60846c422871 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:45.727701 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0948145-1164-42b9-9da5-60846c422871 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:45.727701 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0948145-1164-42b9-9da5-60846c422871 None admin] PATH_INFO: `/v3/groups/2f2aedb71aea48de9f2afd71a3301666` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:45.727701 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0948145-1164-42b9-9da5-60846c422871 None admin] RBAC: Authorizing `identity:get_group(group_id=2f2aedb71aea48de9f2afd71a3301666)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:45.727933 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5c6b46-92c8-4ca1-92df-02d894e3f902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:45.728028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0948145-1164-42b9-9da5-60846c422871 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:45.728336 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c6b46-92c8-4ca1-92df-02d894e3f902 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:45.728618 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c6b46-92c8-4ca1-92df-02d894e3f902 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:45.728890 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5c6b46-92c8-4ca1-92df-02d894e3f902 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:45.729502 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 161/334] 10.222.0.17 () {56 vars in 1180 bytes} [Mon Oct 17 03:32:45 2022] GET /identity/v3/groups/2f2aedb71aea48de9f2afd71a3301666 => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:45.730150 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5c6b46-92c8-4ca1-92df-02d894e3f902 None admin] RBAC: Adding query filter params (group.id=32be849bef724c94a2e14a0e8b62b35d, role.id=609a4bf154ff4dcabb408b3789d28983, scope.project.id=9a1236f1bb0f4a499c0206128e5c97f9) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:45.730432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5c6b46-92c8-4ca1-92df-02d894e3f902 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:45.731859 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5c6b46-92c8-4ca1-92df-02d894e3f902 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:45.735685 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0948145-1164-42b9-9da5-60846c422871 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:45.737669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 174/335] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Oct 17 03:32:45 2022] GET /identity/v3/role_assignments?group.id=32be849bef724c94a2e14a0e8b62b35d&scope.project.id=9a1236f1bb0f4a499c0206128e5c97f9&role.id=609a4bf154ff4dcabb408b3789d28983 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:45.805551 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:45.842364 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0948145-1164-42b9-9da5-60846c422871 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:45.877945 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d9c479-3d35-4771-92db-761b5d81abdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:45.878567 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9c479-3d35-4771-92db-761b5d81abdf None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:45.878848 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9c479-3d35-4771-92db-761b5d81abdf None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:45.879119 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9c479-3d35-4771-92db-761b5d81abdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:45.879894 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d9c479-3d35-4771-92db-761b5d81abdf None admin] RBAC: Adding query filter params (group.id=2f2aedb71aea48de9f2afd71a3301666, role.id=611120afb3a544d0b5161da1b769a2f8, scope.project.id=6f518d7d57a2406a8345926faf8a2f14) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:45.880205 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d9c479-3d35-4771-92db-761b5d81abdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:45.881658 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d9c479-3d35-4771-92db-761b5d81abdf None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:45.888143 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 162/336] 10.222.0.17 () {56 vars in 1376 bytes} [Mon Oct 17 03:32:45 2022] GET /identity/v3/role_assignments?group.id=2f2aedb71aea48de9f2afd71a3301666&scope.project.id=6f518d7d57a2406a8345926faf8a2f14&role.id=611120afb3a544d0b5161da1b769a2f8 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:32:48.379153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd838390-b47a-4573-a5cb-9643e1a935b6 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:48.379672 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd838390-b47a-4573-a5cb-9643e1a935b6 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:48.379672 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd838390-b47a-4573-a5cb-9643e1a935b6 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:48.381049 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 175/337] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:48.397828 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7803c5-6071-41c3-8645-cdf70978f1aa None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:48.398791 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7803c5-6071-41c3-8645-cdf70978f1aa None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:48.399264 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7803c5-6071-41c3-8645-cdf70978f1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:48.424935 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-9f7803c5-6071-41c3-8645-cdf70978f1aa None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:48.426501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-9f7803c5-6071-41c3-8645-cdf70978f1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:48.517058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 163/338] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:48.573736 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631472d6-d53e-4fd4-bfdb-4d482c2cfc5e None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:48.574346 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631472d6-d53e-4fd4-bfdb-4d482c2cfc5e None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:48.574818 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631472d6-d53e-4fd4-bfdb-4d482c2cfc5e None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:48.576638 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 176/339] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:48.582207 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f13ed2-0dd9-4243-b6ac-9edcd4dea992 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:48.582735 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f13ed2-0dd9-4243-b6ac-9edcd4dea992 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:48.583204 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f13ed2-0dd9-4243-b6ac-9edcd4dea992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:48.602531 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-a4f13ed2-0dd9-4243-b6ac-9edcd4dea992 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:48.604064 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-a4f13ed2-0dd9-4243-b6ac-9edcd4dea992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:48.662761 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:48.674472 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 164/340] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:48.893081 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631472d6-d53e-4fd4-bfdb-4d482c2cfc5e None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:48.905053 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8038eb-0084-46d8-8daa-516e4621bf6c None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:48.907354 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8038eb-0084-46d8-8daa-516e4621bf6c None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:48.907906 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8038eb-0084-46d8-8daa-516e4621bf6c None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:48.910994 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 165/341] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:48.915686 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5e2de-570d-4fb3-9254-82f9d7c328de None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:48.916204 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5e2de-570d-4fb3-9254-82f9d7c328de None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:48.916676 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5e2de-570d-4fb3-9254-82f9d7c328de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:48.947271 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-fbc5e2de-570d-4fb3-9254-82f9d7c328de None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:48.951248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-fbc5e2de-570d-4fb3-9254-82f9d7c328de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:49.041939 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631472d6-d53e-4fd4-bfdb-4d482c2cfc5e None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:49.051805 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 166/342] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 03:32:49.054716 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc5e2de-570d-4fb3-9254-82f9d7c328de None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:49.104499 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa2e8f-cd1c-492c-850f-27366f2dd061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:49.105519 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa2e8f-cd1c-492c-850f-27366f2dd061 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:49.107151 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa2e8f-cd1c-492c-850f-27366f2dd061 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:49.107625 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa2e8f-cd1c-492c-850f-27366f2dd061 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:49.133304 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa2e8f-cd1c-492c-850f-27366f2dd061 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:49.135101 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa2e8f-cd1c-492c-850f-27366f2dd061 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:49.153456 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-e3fa2e8f-cd1c-492c-850f-27366f2dd061 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:49.153456 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:49.153456 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:49.153456 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:49.153456 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:49.153456 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:49.153456 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:49.153456 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:49.153456 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:49.153456 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:49.153456 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:49.153456 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:49.153456 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:32:49.153456 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:32:49.153456 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:32:49.153456 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:32:49.154401 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:49.154401 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:49.154401 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:49.154401 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:49.154401 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:49.154401 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:49.154401 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:49.154401 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:49.154401 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:49.154401 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:49.154401 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:49.154401 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:49.154401 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:49.154401 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:49.154401 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:49.154401 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:49.154401 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:32:49.155229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:32:49.155229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:49.155229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:49.155229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 177/343] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:32:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:49.161391 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa2e8f-cd1c-492c-850f-27366f2dd061 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:49.242127 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc5e2de-570d-4fb3-9254-82f9d7c328de None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:49.254762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:49.287773 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:49.288701 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac18671-75be-457c-9b8c-69e8093669c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:49.289640 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac18671-75be-457c-9b8c-69e8093669c8 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:49.290173 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac18671-75be-457c-9b8c-69e8093669c8 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:49.290647 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac18671-75be-457c-9b8c-69e8093669c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:49.311713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac18671-75be-457c-9b8c-69e8093669c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:49.313531 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac18671-75be-457c-9b8c-69e8093669c8 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:49.319778 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa2e8f-cd1c-492c-850f-27366f2dd061 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:49.344054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-8ac18671-75be-457c-9b8c-69e8093669c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:49.344054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:49.344054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:49.344054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:49.344054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:49.344054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:49.344054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:49.344054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:49.344054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:49.344054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:49.344054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:49.344054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:49.344054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:32:49.344054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:32:49.344054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:32:49.344054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:32:49.345018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:49.345018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:49.345018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:49.345018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:49.345018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:49.345018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:49.345018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:49.345018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:49.345018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:49.345018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:49.345018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:49.345018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:49.345018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:49.345018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:49.345018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:49.345018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:49.345018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:32:49.345866 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:32:49.345866 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:49.345866 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:49.354684 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64a649-ecdd-4dc2-9f31-7ea93e8a0ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:49.355124 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64a649-ecdd-4dc2-9f31-7ea93e8a0ebd None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:49.355406 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64a649-ecdd-4dc2-9f31-7ea93e8a0ebd None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:49.355681 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64a649-ecdd-4dc2-9f31-7ea93e8a0ebd None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:49.356129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 167/344] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:32:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:49.356388 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64a649-ecdd-4dc2-9f31-7ea93e8a0ebd None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:49.356695 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64a649-ecdd-4dc2-9f31-7ea93e8a0ebd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:49.358100 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64a649-ecdd-4dc2-9f31-7ea93e8a0ebd None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:49.360643 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac18671-75be-457c-9b8c-69e8093669c8 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:49.372313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 178/345] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:49.380742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64a649-ecdd-4dc2-9f31-7ea93e8a0ebd None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:49.499832 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac18671-75be-457c-9b8c-69e8093669c8 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:49.515453 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64a649-ecdd-4dc2-9f31-7ea93e8a0ebd None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:49.518565 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:49.534077 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42065b83-5880-4367-922f-79a6f9c23666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:49.534689 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42065b83-5880-4367-922f-79a6f9c23666 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:49.534973 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42065b83-5880-4367-922f-79a6f9c23666 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:49.535363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42065b83-5880-4367-922f-79a6f9c23666 None admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:49.536042 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42065b83-5880-4367-922f-79a6f9c23666 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:49.536345 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42065b83-5880-4367-922f-79a6f9c23666 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:49.537701 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42065b83-5880-4367-922f-79a6f9c23666 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:49.551638 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 168/346] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:49.552382 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42065b83-5880-4367-922f-79a6f9c23666 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:49.554247 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daede6b7-c2b4-4b78-aef3-0f285b522db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:49.554865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daede6b7-c2b4-4b78-aef3-0f285b522db9 None admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:49.555148 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daede6b7-c2b4-4b78-aef3-0f285b522db9 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:49.555425 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daede6b7-c2b4-4b78-aef3-0f285b522db9 None admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:49.556170 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daede6b7-c2b4-4b78-aef3-0f285b522db9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:49.558235 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daede6b7-c2b4-4b78-aef3-0f285b522db9 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:49.586744 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:49.642073 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 179/347] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Oct 17 03:32:49 2022] POST /identity/v3/users => generated 246 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:32:49.643324 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72fd0e-a969-4651-b97e-71d3c2a199a5 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:49.643610 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72fd0e-a969-4651-b97e-71d3c2a199a5 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:49.643906 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72fd0e-a969-4651-b97e-71d3c2a199a5 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:49.645288 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 180/348] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:49.645818 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f72fd0e-a969-4651-b97e-71d3c2a199a5 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:49.674318 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42065b83-5880-4367-922f-79a6f9c23666 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:49.725324 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e59d4d-3b6c-436e-95de-0c47f77922b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:49.726165 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e59d4d-3b6c-436e-95de-0c47f77922b4 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:49.726577 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e59d4d-3b6c-436e-95de-0c47f77922b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:49.726867 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e59d4d-3b6c-436e-95de-0c47f77922b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:49.742943 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e59d4d-3b6c-436e-95de-0c47f77922b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:49.744800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e59d4d-3b6c-436e-95de-0c47f77922b4 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:49.771155 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-d9e59d4d-3b6c-436e-95de-0c47f77922b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:49.771155 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:49.771155 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:49.771155 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:49.771155 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:49.771155 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:49.771155 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:49.771155 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:49.771155 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:49.771155 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:49.771155 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:49.771155 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:49.771155 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:32:49.771155 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:32:49.771155 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:32:49.771155 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:32:49.772094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:49.772094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:49.772094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:49.772094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:49.772094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:49.772094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:49.772094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:49.772094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:49.772094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:49.772094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:49.772094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:49.772094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:49.772094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:49.772094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:49.772094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:49.772094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:49.772094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:32:49.772925 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:32:49.772925 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:49.772925 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:49.775893 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 169/349] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:32:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:49.777665 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44a040-e205-4eb1-976c-3d501ab94916 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:49.778477 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44a040-e205-4eb1-976c-3d501ab94916 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:49.778751 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44a040-e205-4eb1-976c-3d501ab94916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:49.795588 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f72fd0e-a969-4651-b97e-71d3c2a199a5 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:49.802042 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-fa44a040-e205-4eb1-976c-3d501ab94916 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:49.803292 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-fa44a040-e205-4eb1-976c-3d501ab94916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:49.844265 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee8e6f5-308e-4018-b71e-85891532235e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:49.845846 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee8e6f5-308e-4018-b71e-85891532235e None admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:49.846129 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee8e6f5-308e-4018-b71e-85891532235e None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:49.846400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee8e6f5-308e-4018-b71e-85891532235e None admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:49.847049 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee8e6f5-308e-4018-b71e-85891532235e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:49.849400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee8e6f5-308e-4018-b71e-85891532235e None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:49.887839 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 170/350] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:49.888794 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44a040-e205-4eb1-976c-3d501ab94916 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:49.905425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:49.957739 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 181/351] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Oct 17 03:32:49 2022] POST /identity/v3/users => generated 248 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:32:50.036422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44a040-e205-4eb1-976c-3d501ab94916 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:50.087141 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d5c264-b5b2-4fb6-a5c2-f570c26cfa1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:50.087797 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5c264-b5b2-4fb6-a5c2-f570c26cfa1b None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:50.088192 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5c264-b5b2-4fb6-a5c2-f570c26cfa1b None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:50.088472 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d5c264-b5b2-4fb6-a5c2-f570c26cfa1b None admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:50.089679 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d5c264-b5b2-4fb6-a5c2-f570c26cfa1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:50.089987 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d5c264-b5b2-4fb6-a5c2-f570c26cfa1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:50.091647 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d5c264-b5b2-4fb6-a5c2-f570c26cfa1b None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:50.111682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 171/352] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:32:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:50.117597 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee8e6f5-308e-4018-b71e-85891532235e None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:50.144406 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:50.233609 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f10897-1cb3-435a-adce-809fa2ab308c None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:50.233947 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee8e6f5-308e-4018-b71e-85891532235e None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:50.234051 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f10897-1cb3-435a-adce-809fa2ab308c None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:50.234201 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f10897-1cb3-435a-adce-809fa2ab308c None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:50.235913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 172/353] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:50.242431 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01715d61-fe94-4aa6-8546-adf1cdee32ed None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:50.242720 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01715d61-fe94-4aa6-8546-adf1cdee32ed None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:50.242995 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01715d61-fe94-4aa6-8546-adf1cdee32ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:50.269018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-01715d61-fe94-4aa6-8546-adf1cdee32ed None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:50.270488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-01715d61-fe94-4aa6-8546-adf1cdee32ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:50.274341 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500cd765-011e-4ad9-be3f-2811ce128c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:50.274968 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500cd765-011e-4ad9-be3f-2811ce128c1e None admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:50.275246 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500cd765-011e-4ad9-be3f-2811ce128c1e None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:50.275522 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500cd765-011e-4ad9-be3f-2811ce128c1e None admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:50.276790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500cd765-011e-4ad9-be3f-2811ce128c1e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:50.278197 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500cd765-011e-4ad9-be3f-2811ce128c1e None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:50.350545 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 173/354] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 03:32:50.351081 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01715d61-fe94-4aa6-8546-adf1cdee32ed None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:50.355200 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:50.374538 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 182/355] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Oct 17 03:32:50 2022] POST /identity/v3/users => generated 248 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:32:50.476036 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01715d61-fe94-4aa6-8546-adf1cdee32ed None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:50.523329 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7a98e-ccc1-4bc4-8f82-f607bd351899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:50.524117 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7a98e-ccc1-4bc4-8f82-f607bd351899 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:50.524330 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7a98e-ccc1-4bc4-8f82-f607bd351899 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:50.524640 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7a98e-ccc1-4bc4-8f82-f607bd351899 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:50.538592 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7a98e-ccc1-4bc4-8f82-f607bd351899 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:50.540159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7a98e-ccc1-4bc4-8f82-f607bd351899 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:50.561430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-85e7a98e-ccc1-4bc4-8f82-f607bd351899 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:50.561430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:50.561430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:50.561430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:50.561430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:50.561430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:50.561430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:50.561430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:50.561430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:50.561430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:50.561430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:50.561430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:50.561430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:32:50.561430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:32:50.561430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:32:50.561430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:32:50.562371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:50.562371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:50.562371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:50.562371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:50.562371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:50.562371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:50.562371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:50.562371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:50.562371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:50.562371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:50.562371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:50.562371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:50.562371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:50.562371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:50.562371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:50.562371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:50.562371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:32:50.564276 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:32:50.564276 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:50.564276 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:50.564276 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 174/356] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:32:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:50.568850 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500cd765-011e-4ad9-be3f-2811ce128c1e None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:50.670920 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:50.707270 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500cd765-011e-4ad9-be3f-2811ce128c1e None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:50.744539 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff0f6af-5c21-44a5-a0f6-9a00b13eaee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:50.746021 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0f6af-5c21-44a5-a0f6-9a00b13eaee5 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:50.746444 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0f6af-5c21-44a5-a0f6-9a00b13eaee5 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:50.746857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0f6af-5c21-44a5-a0f6-9a00b13eaee5 None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:50.747648 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff0f6af-5c21-44a5-a0f6-9a00b13eaee5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:50.748137 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff0f6af-5c21-44a5-a0f6-9a00b13eaee5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:50.749728 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff0f6af-5c21-44a5-a0f6-9a00b13eaee5 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:50.764069 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 183/357] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:50.770467 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7a98e-ccc1-4bc4-8f82-f607bd351899 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:50.779453 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff0f6af-5c21-44a5-a0f6-9a00b13eaee5 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:50.845271 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:50.909954 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff0f6af-5c21-44a5-a0f6-9a00b13eaee5 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:50.922025 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7a98e-ccc1-4bc4-8f82-f607bd351899 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:50.944765 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:50.951411 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a826a-bd89-4c02-961f-53b2c3fd5e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:50.952131 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a826a-bd89-4c02-961f-53b2c3fd5e17 None admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:50.958374 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a826a-bd89-4c02-961f-53b2c3fd5e17 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:50.958374 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a826a-bd89-4c02-961f-53b2c3fd5e17 None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:50.958374 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78a826a-bd89-4c02-961f-53b2c3fd5e17 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:50.958374 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78a826a-bd89-4c02-961f-53b2c3fd5e17 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:50.983599 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9265559-2dea-4a81-985a-cfbc1eadff5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:50.984508 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9265559-2dea-4a81-985a-cfbc1eadff5a None admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:50.984936 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9265559-2dea-4a81-985a-cfbc1eadff5a None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:50.985414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9265559-2dea-4a81-985a-cfbc1eadff5a None admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:50.986374 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9265559-2dea-4a81-985a-cfbc1eadff5a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:50.987983 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9265559-2dea-4a81-985a-cfbc1eadff5a None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:51.023167 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.sql.core [None req-e78a826a-bd89-4c02-961f-53b2c3fd5e17 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 17 03:32:51.023167 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 17 03:32:51.023167 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [parameters: {'id': 'fe64be5939df492ebaab3c23de0f3350', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 17 03:32:51.023167 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88300) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 03:32:51.029865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-e78a826a-bd89-4c02-961f-53b2c3fd5e17 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 17 03:32:51.029865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:51.029865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 03:32:51.029865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 03:32:51.029865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 03:32:51.029865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 03:32:51.029865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 03:32:51.029865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 03:32:51.029865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 03:32:51.029865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application conn.query(q) Oct 17 03:32:51.029865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 03:32:51.029865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 03:32:51.029865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 03:32:51.029865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application result.read() Oct 17 03:32:51.029865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 03:32:51.036444 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 03:32:51.036444 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 03:32:51.036444 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 03:32:51.036444 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 03:32:51.036444 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 03:32:51.036444 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 03:32:51.036444 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 03:32:51.036444 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 17 03:32:51.036444 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:51.036444 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 03:32:51.036444 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:51.036444 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:51.036444 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 03:32:51.036444 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 03:32:51.036444 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 17 03:32:51.036444 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return ref.to_dict() Oct 17 03:32:51.036444 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 03:32:51.036444 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application next(self.gen) Oct 17 03:32:51.037582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 03:32:51.037582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application yield resource Oct 17 03:32:51.037582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 03:32:51.037582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application next(self.gen) Oct 17 03:32:51.037582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 03:32:51.037582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 03:32:51.037582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 03:32:51.037582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 03:32:51.037582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 03:32:51.037582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise self.value Oct 17 03:32:51.037582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 03:32:51.037582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 03:32:51.037582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 03:32:51.037582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application session.commit() Oct 17 03:32:51.037582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 03:32:51.037582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 03:32:51.037582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 03:32:51.038654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 03:32:51.038654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 03:32:51.038654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self.session.flush() Oct 17 03:32:51.038654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 03:32:51.038654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 03:32:51.038654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 03:32:51.038654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 03:32:51.038654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 03:32:51.038654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application compat.raise_( Oct 17 03:32:51.038654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 03:32:51.038654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception Oct 17 03:32:51.038654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 03:32:51.038654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 03:32:51.038654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 03:32:51.038654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 03:32:51.038654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 03:32:51.038654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 03:32:51.039678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 03:32:51.039678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 03:32:51.039678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 17 03:32:51.039678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 17 03:32:51.039678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 03:32:51.039678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 03:32:51.039678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 03:32:51.039678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 03:32:51.039678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 03:32:51.039678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 03:32:51.039678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 03:32:51.039678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 03:32:51.039678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 03:32:51.039678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 03:32:51.039678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 03:32:51.039678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception Oct 17 03:32:51.040814 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 03:32:51.040814 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 03:32:51.040814 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 03:32:51.040814 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 03:32:51.040814 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 03:32:51.040814 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 03:32:51.040814 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 03:32:51.040814 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application conn.query(q) Oct 17 03:32:51.040814 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 03:32:51.040814 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 03:32:51.040814 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 03:32:51.040814 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application result.read() Oct 17 03:32:51.040814 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 03:32:51.040814 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 03:32:51.040814 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 03:32:51.040814 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 03:32:51.042280 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 03:32:51.042280 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 03:32:51.042280 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 03:32:51.042280 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 03:32:51.042280 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 17 03:32:51.042280 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 17 03:32:51.042280 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [parameters: {'id': 'fe64be5939df492ebaab3c23de0f3350', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 17 03:32:51.042280 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 03:32:51.042280 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:51.042280 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:32:51.042280 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:51.042280 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:51.042280 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:51.042280 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:51.042280 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:51.042280 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:51.043143 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:51.043143 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:51.043143 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:51.043143 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:51.043143 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:51.043143 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:51.043143 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 17 03:32:51.043143 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 17 03:32:51.043143 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:51.043143 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:51.043143 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 17 03:32:51.043143 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 17 03:32:51.043143 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 03:32:51.043143 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 03:32:51.043143 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 17 03:32:51.043143 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:51.044032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 184/358] 10.222.0.17 () {60 vars in 1131 bytes} [Mon Oct 17 03:32:50 2022] POST /identity/v3/roles => generated 144 bytes in 252 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:51.046796 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a826a-bd89-4c02-961f-53b2c3fd5e17 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:51.099295 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 175/359] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Oct 17 03:32:50 2022] POST /identity/v3/users => generated 249 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:32:51.179204 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a826a-bd89-4c02-961f-53b2c3fd5e17 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:51.220215 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2818b9c7-6aef-4a7b-8ac1-d4a0ad9ecea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:51.221257 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818b9c7-6aef-4a7b-8ac1-d4a0ad9ecea1 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:51.221794 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818b9c7-6aef-4a7b-8ac1-d4a0ad9ecea1 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:51.222241 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818b9c7-6aef-4a7b-8ac1-d4a0ad9ecea1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:51.234223 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2818b9c7-6aef-4a7b-8ac1-d4a0ad9ecea1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:51.239806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-2818b9c7-6aef-4a7b-8ac1-d4a0ad9ecea1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 03:32:51.239806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:51.239806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:51.239806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:51.239806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:51.239806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:51.239806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:51.239806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:51.239806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:51.239806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:51.239806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:51.239806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:51.239806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:51.239806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:51.239806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:51.239806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:51.240750 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:51.240750 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:51.240750 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:51.240750 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:51.240750 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:51.240750 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:51.240750 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:51.240750 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:51.240750 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:51.240750 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:51.240750 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:51.240750 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:51.240750 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:51.240750 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:51.240750 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:51.240750 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:51.241656 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:51.241656 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:51.241656 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:51.241656 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:51.241656 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:51.241656 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:51.241656 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:51.241656 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:51.241656 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:51.241656 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:51.241656 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 17 03:32:51.241656 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:51.243778 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 185/360] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Oct 17 03:32:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:32:51.250611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9265559-2dea-4a81-985a-cfbc1eadff5a None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:51.379031 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9265559-2dea-4a81-985a-cfbc1eadff5a None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:51.393468 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:51.417660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37f476-f277-44a3-bb9a-04c0ecb4f471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:51.418279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37f476-f277-44a3-bb9a-04c0ecb4f471 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:51.418559 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37f476-f277-44a3-bb9a-04c0ecb4f471 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:51.418827 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37f476-f277-44a3-bb9a-04c0ecb4f471 None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:51.419494 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a37f476-f277-44a3-bb9a-04c0ecb4f471 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:51.419820 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a37f476-f277-44a3-bb9a-04c0ecb4f471 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:51.421191 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a37f476-f277-44a3-bb9a-04c0ecb4f471 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:51.430046 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 176/361] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:32:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:52.046016 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d8a2a-8074-48ee-9d34-cfbff981e015 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:52.046633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d8a2a-8074-48ee-9d34-cfbff981e015 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:52.046633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d8a2a-8074-48ee-9d34-cfbff981e015 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:52.048135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 186/362] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:52.053476 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff2406-3653-4c97-84ce-21b4be693805 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:52.053901 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff2406-3653-4c97-84ce-21b4be693805 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:52.054094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff2406-3653-4c97-84ce-21b4be693805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:52.072387 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-27ff2406-3653-4c97-84ce-21b4be693805 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:52.074096 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-27ff2406-3653-4c97-84ce-21b4be693805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:52.157251 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 177/363] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:52.435384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d8a2a-8074-48ee-9d34-cfbff981e015 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:52.511568 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:52.555210 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d8a2a-8074-48ee-9d34-cfbff981e015 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:52.590514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683b5fc1-350a-4e2d-a567-6c1b323e9ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:52.591147 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b5fc1-350a-4e2d-a567-6c1b323e9ebc None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:52.591423 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b5fc1-350a-4e2d-a567-6c1b323e9ebc None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:52.591692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683b5fc1-350a-4e2d-a567-6c1b323e9ebc None admin] PATH_INFO: `/v3/roles/service` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:52.601974 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683b5fc1-350a-4e2d-a567-6c1b323e9ebc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:52.608433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-683b5fc1-350a-4e2d-a567-6c1b323e9ebc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:32:52.608433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:52.608433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:52.608433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:52.608433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:52.608433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:52.608433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:52.608433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:52.608433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:52.608433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:52.608433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:52.608433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:52.608433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:52.608433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:52.608433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:52.608433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:52.609382 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:52.609382 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:52.609382 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:52.609382 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:52.609382 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:52.609382 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:52.609382 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:52.609382 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:52.609382 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:52.609382 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:52.609382 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:52.609382 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:52.609382 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:52.609382 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:52.609382 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:52.609382 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:52.610206 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:52.610206 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:52.610206 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:52.610206 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:52.610206 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:52.610206 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:52.610206 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:52.610206 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:52.610206 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:52.610206 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:52.610206 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:32:52.610206 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:52.614422 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 187/364] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Oct 17 03:32:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:52.615794 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff2406-3653-4c97-84ce-21b4be693805 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:52.759626 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff2406-3653-4c97-84ce-21b4be693805 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:52.762126 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:52.789617 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91c1ba-e17a-4438-954c-a47e4e808599 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:52.789951 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91c1ba-e17a-4438-954c-a47e4e808599 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:52.790228 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91c1ba-e17a-4438-954c-a47e4e808599 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:52.791589 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 188/365] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:32:52.798287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9248853-8382-4864-aae0-9cc6872a5f34 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:52.798670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9248853-8382-4864-aae0-9cc6872a5f34 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:52.798927 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9248853-8382-4864-aae0-9cc6872a5f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:52.814050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b783e-6a8d-4848-a6db-6b3af5bb2dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:52.814665 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b783e-6a8d-4848-a6db-6b3af5bb2dd7 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:52.814940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b783e-6a8d-4848-a6db-6b3af5bb2dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:52.815209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140b783e-6a8d-4848-a6db-6b3af5bb2dd7 None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:52.815897 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b783e-6a8d-4848-a6db-6b3af5bb2dd7 None admin] RBAC: Adding query filter params (name=service) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:52.816202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b783e-6a8d-4848-a6db-6b3af5bb2dd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:52.817557 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140b783e-6a8d-4848-a6db-6b3af5bb2dd7 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:52.828471 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-f9248853-8382-4864-aae0-9cc6872a5f34 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:52.829718 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-f9248853-8382-4864-aae0-9cc6872a5f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:52.834531 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 178/366] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Oct 17 03:32:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:52.841810 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b783e-6a8d-4848-a6db-6b3af5bb2dd7 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:52.973009 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 189/367] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 03:32:53.006549 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140b783e-6a8d-4848-a6db-6b3af5bb2dd7 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:53.040395 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba3fb66-8a0b-4164-ba6d-25cb337ab96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:53.041007 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3fb66-8a0b-4164-ba6d-25cb337ab96b None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:53.041293 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3fb66-8a0b-4164-ba6d-25cb337ab96b None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:53.041567 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba3fb66-8a0b-4164-ba6d-25cb337ab96b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:53.052881 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba3fb66-8a0b-4164-ba6d-25cb337ab96b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:53.054372 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba3fb66-8a0b-4164-ba6d-25cb337ab96b None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:53.069435 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-cba3fb66-8a0b-4164-ba6d-25cb337ab96b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:53.069435 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:53.069435 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:53.069435 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:53.069435 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:53.069435 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:53.069435 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:53.069435 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:53.069435 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:53.069435 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:53.069435 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:53.069435 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:53.069435 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:32:53.069435 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:32:53.069435 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:32:53.069435 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:32:53.070334 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:53.070334 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:53.070334 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:53.070334 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:53.070334 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:53.070334 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:53.070334 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:53.070334 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:53.070334 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:53.070334 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:53.070334 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:53.070334 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:53.070334 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:53.070334 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:53.070334 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:53.070334 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:53.070334 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:32:53.071177 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:32:53.071177 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:53.071177 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:53.075831 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 179/368] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:32:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:53.077207 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9248853-8382-4864-aae0-9cc6872a5f34 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:53.137364 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:53.211820 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9248853-8382-4864-aae0-9cc6872a5f34 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:53.260093 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0828fe-a244-4b39-99ff-54d776985e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:53.260737 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0828fe-a244-4b39-99ff-54d776985e91 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:53.261014 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0828fe-a244-4b39-99ff-54d776985e91 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:53.261287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0828fe-a244-4b39-99ff-54d776985e91 None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:53.261927 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0828fe-a244-4b39-99ff-54d776985e91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:53.262227 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0828fe-a244-4b39-99ff-54d776985e91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:53.263573 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0828fe-a244-4b39-99ff-54d776985e91 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:53.268965 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9bb1a-c941-4376-88fb-1804c1a1ef08 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:53.269255 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9bb1a-c941-4376-88fb-1804c1a1ef08 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:53.269527 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a9bb1a-c941-4376-88fb-1804c1a1ef08 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:53.270869 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 180/369] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:32:53.275699 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d51d2-c9b4-47ed-b5b8-24b650d1c2dd None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:53.276048 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d51d2-c9b4-47ed-b5b8-24b650d1c2dd None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:53.276323 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d51d2-c9b4-47ed-b5b8-24b650d1c2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:53.308077 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 190/370] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:32:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:53.308985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-493d51d2-c9b4-47ed-b5b8-24b650d1c2dd None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:53.310185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-493d51d2-c9b4-47ed-b5b8-24b650d1c2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:53.326959 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0828fe-a244-4b39-99ff-54d776985e91 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:53.331485 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:53.413705 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 181/371] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:53.415053 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d51d2-c9b4-47ed-b5b8-24b650d1c2dd None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:53.446159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0828fe-a244-4b39-99ff-54d776985e91 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:53.455181 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:53.498507 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c5473-334b-472e-8b30-c895ed1e7af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:53.499143 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c5473-334b-472e-8b30-c895ed1e7af4 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:53.499416 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c5473-334b-472e-8b30-c895ed1e7af4 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:53.499683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2c5473-334b-472e-8b30-c895ed1e7af4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:53.511902 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c5473-334b-472e-8b30-c895ed1e7af4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:53.513322 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2c5473-334b-472e-8b30-c895ed1e7af4 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:53.535903 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-8f2c5473-334b-472e-8b30-c895ed1e7af4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 03:32:53.535903 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:53.535903 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:53.535903 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:53.535903 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:53.535903 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:53.535903 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:53.535903 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:53.535903 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:53.535903 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:53.535903 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:53.535903 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:53.535903 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:32:53.535903 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:32:53.535903 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:32:53.535903 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:32:53.536894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:53.536894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:53.536894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:32:53.536894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:32:53.536894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:32:53.536894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:32:53.536894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:53.536894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:53.536894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:53.536894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:53.536894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:53.536894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:53.536894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:53.536894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:53.536894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:53.536894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:53.536894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:53.536894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:53.537941 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:53.537941 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:53.537941 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:32:53.537941 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:32:53.537941 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:32:53.537941 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:32:53.537941 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:32:53.537941 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:32:53.537941 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 03:32:53.537941 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:53.537941 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 191/372] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Oct 17 03:32:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:32:53.545009 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c5473-334b-472e-8b30-c895ed1e7af4 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:53.573964 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d51d2-c9b4-47ed-b5b8-24b650d1c2dd None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:53.657012 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2c5473-334b-472e-8b30-c895ed1e7af4 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:53.658024 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d34d9-2941-45fa-ac06-56500c7477a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:53.658949 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d34d9-2941-45fa-ac06-56500c7477a7 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:53.659416 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d34d9-2941-45fa-ac06-56500c7477a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:53.659901 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076d34d9-2941-45fa-ac06-56500c7477a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:53.680353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076d34d9-2941-45fa-ac06-56500c7477a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:53.687183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-076d34d9-2941-45fa-ac06-56500c7477a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:32:53.687183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:53.687183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:53.687183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:53.687183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:53.687183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:53.687183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:53.687183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:53.687183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:53.687183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:53.687183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:53.687183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:53.687183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:53.687183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:53.687183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:53.687183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:53.688083 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:53.688083 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:53.688083 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:53.688083 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:53.688083 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:53.688083 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:53.688083 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:53.688083 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:53.688083 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:53.688083 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:53.688083 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:53.688083 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:53.688083 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:53.688083 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:53.688083 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:53.688083 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:53.688855 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:53.688855 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:53.688855 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:53.688855 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:53.688855 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:53.688855 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:53.688855 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:53.688855 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:53.688855 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:53.688855 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:53.688855 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:32:53.688855 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:53.695021 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 182/373] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Oct 17 03:32:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:53.708780 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d34d9-2941-45fa-ac06-56500c7477a7 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:53.719759 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f074ef-19ab-4805-9daf-b5cd1c7f92f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:53.719759 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f074ef-19ab-4805-9daf-b5cd1c7f92f2 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:53.720015 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f074ef-19ab-4805-9daf-b5cd1c7f92f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:53.720096 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f074ef-19ab-4805-9daf-b5cd1c7f92f2 None admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:53.720804 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f074ef-19ab-4805-9daf-b5cd1c7f92f2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:53.721107 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f074ef-19ab-4805-9daf-b5cd1c7f92f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:53.723607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f074ef-19ab-4805-9daf-b5cd1c7f92f2 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:53.733640 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:53.761973 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 192/374] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Oct 17 03:32:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:53.770039 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f074ef-19ab-4805-9daf-b5cd1c7f92f2 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:53.816055 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:53.873563 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076d34d9-2941-45fa-ac06-56500c7477a7 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:53.909283 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f074ef-19ab-4805-9daf-b5cd1c7f92f2 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:53.919251 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db39f3-7b1a-4557-9b7a-fb104a894d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:53.920241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db39f3-7b1a-4557-9b7a-fb104a894d51 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:53.920734 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db39f3-7b1a-4557-9b7a-fb104a894d51 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:53.921203 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db39f3-7b1a-4557-9b7a-fb104a894d51 None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:53.922097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db39f3-7b1a-4557-9b7a-fb104a894d51 None admin] RBAC: Adding query filter params (name=service) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:53.922591 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db39f3-7b1a-4557-9b7a-fb104a894d51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:53.924229 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db39f3-7b1a-4557-9b7a-fb104a894d51 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:53.934280 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 183/375] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Oct 17 03:32:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:53.935947 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db39f3-7b1a-4557-9b7a-fb104a894d51 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:53.951161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda18cc-8649-4624-b9b0-947278360805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:53.951855 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda18cc-8649-4624-b9b0-947278360805 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:53.952173 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda18cc-8649-4624-b9b0-947278360805 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:53.952449 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda18cc-8649-4624-b9b0-947278360805 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:53.963045 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda18cc-8649-4624-b9b0-947278360805 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:53.965304 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda18cc-8649-4624-b9b0-947278360805 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:53.981171 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-7fda18cc-8649-4624-b9b0-947278360805 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:53.981171 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:53.981171 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:53.981171 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:53.981171 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:53.981171 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:53.981171 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:53.981171 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:53.981171 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:53.981171 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:53.981171 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:53.981171 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:53.981171 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:32:53.981171 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:32:53.981171 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:32:53.981171 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:32:53.982091 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:53.982091 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:53.982091 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:53.982091 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:53.982091 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:53.982091 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:53.982091 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:53.982091 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:53.982091 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:53.982091 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:53.982091 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:53.982091 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:53.982091 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:53.982091 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:53.982091 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:53.982091 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:53.982091 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:32:53.982905 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:32:53.982905 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:53.982905 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:53.986888 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 193/376] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:32:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:32:53.987725 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda18cc-8649-4624-b9b0-947278360805 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:54.017431 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:54.050854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db39f3-7b1a-4557-9b7a-fb104a894d51 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:54.074954 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:54.088426 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fffdaa4-81e2-47e5-8374-e77ea56cf556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:54.089603 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffdaa4-81e2-47e5-8374-e77ea56cf556 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:54.090314 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffdaa4-81e2-47e5-8374-e77ea56cf556 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:54.090425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffdaa4-81e2-47e5-8374-e77ea56cf556 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:54.098568 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda18cc-8649-4624-b9b0-947278360805 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:54.101030 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fffdaa4-81e2-47e5-8374-e77ea56cf556 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:54.106602 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-4fffdaa4-81e2-47e5-8374-e77ea56cf556 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:32:54.106602 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:54.106602 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:54.106602 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:54.106602 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:54.106602 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:54.106602 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:54.106602 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:54.106602 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:54.106602 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:54.106602 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:54.106602 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:54.106602 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:54.106602 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:54.106602 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:54.106602 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:54.107459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:54.107459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:54.107459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:54.107459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:54.107459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:54.107459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:54.107459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:54.107459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:54.107459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:54.107459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:54.107459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:54.107459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:54.107459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:54.107459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:54.107459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:54.107459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:54.108317 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:54.108317 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:54.108317 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:54.108317 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:54.108317 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:54.108317 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:54.108317 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:54.108317 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:54.108317 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:54.108317 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:54.108317 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:32:54.108317 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:54.108317 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 184/377] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Oct 17 03:32:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:54.108970 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fffdaa4-81e2-47e5-8374-e77ea56cf556 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:54.139959 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cacc82-2153-4397-bbc1-1061fe3bd143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:54.140518 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cacc82-2153-4397-bbc1-1061fe3bd143 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:54.140797 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cacc82-2153-4397-bbc1-1061fe3bd143 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:54.141067 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cacc82-2153-4397-bbc1-1061fe3bd143 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:54.153128 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cacc82-2153-4397-bbc1-1061fe3bd143 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:54.154574 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cacc82-2153-4397-bbc1-1061fe3bd143 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:54.169721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-a9cacc82-2153-4397-bbc1-1061fe3bd143 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:54.169721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:54.169721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:54.169721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:54.169721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:54.169721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:54.169721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:54.169721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:54.169721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:54.169721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:54.169721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:54.169721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:54.169721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:32:54.169721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:32:54.169721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:32:54.169721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:32:54.170584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:54.170584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:54.170584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:54.170584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:54.170584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:54.170584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:54.170584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:54.170584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:54.170584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:54.170584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:54.170584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:54.170584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:54.170584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:54.170584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:54.170584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:54.170584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:54.170584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:32:54.172491 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:32:54.172491 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:54.172491 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:54.172491 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 194/378] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:32:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:54.172491 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5e2ad-e4d5-4383-9baa-eaa6c26a5d9a None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:54.172491 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5e2ad-e4d5-4383-9baa-eaa6c26a5d9a None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:54.172491 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe5e2ad-e4d5-4383-9baa-eaa6c26a5d9a None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:54.173845 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 195/379] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:54.176986 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5e2ad-e4d5-4383-9baa-eaa6c26a5d9a None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:54.224620 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fffdaa4-81e2-47e5-8374-e77ea56cf556 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:54.254307 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:54.260839 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9bfc88-c2b4-47a1-b59d-e81d8281823b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:54.261452 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bfc88-c2b4-47a1-b59d-e81d8281823b None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:54.261730 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bfc88-c2b4-47a1-b59d-e81d8281823b None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:54.262001 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bfc88-c2b4-47a1-b59d-e81d8281823b None admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:54.262643 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9bfc88-c2b4-47a1-b59d-e81d8281823b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:54.262936 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9bfc88-c2b4-47a1-b59d-e81d8281823b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:54.264318 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9bfc88-c2b4-47a1-b59d-e81d8281823b None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:54.276921 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 185/380] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:54.282022 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9bfc88-c2b4-47a1-b59d-e81d8281823b None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:54.290487 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe5e2ad-e4d5-4383-9baa-eaa6c26a5d9a None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:54.319055 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:54.328311 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea644ea8-854b-4db0-87ff-a5fe161221c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:54.328911 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea644ea8-854b-4db0-87ff-a5fe161221c5 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:54.329300 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea644ea8-854b-4db0-87ff-a5fe161221c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:54.329591 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea644ea8-854b-4db0-87ff-a5fe161221c5 None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:54.330256 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea644ea8-854b-4db0-87ff-a5fe161221c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:54.330556 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea644ea8-854b-4db0-87ff-a5fe161221c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:54.331927 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea644ea8-854b-4db0-87ff-a5fe161221c5 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:54.341016 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 196/381] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Oct 17 03:32:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:54.342335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e1f6c-20f7-46bc-b70a-a7eee04ef4e7 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:54.342538 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e1f6c-20f7-46bc-b70a-a7eee04ef4e7 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:54.342811 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e1f6c-20f7-46bc-b70a-a7eee04ef4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:54.359566 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-b94e1f6c-20f7-46bc-b70a-a7eee04ef4e7 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:54.360773 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-b94e1f6c-20f7-46bc-b70a-a7eee04ef4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:54.385912 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9bfc88-c2b4-47a1-b59d-e81d8281823b None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:54.419503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a643da-583a-4df4-9380-4c6a72420605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:54.420067 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a643da-583a-4df4-9380-4c6a72420605 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:54.420349 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a643da-583a-4df4-9380-4c6a72420605 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:54.420620 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a643da-583a-4df4-9380-4c6a72420605 None admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:54.421254 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a643da-583a-4df4-9380-4c6a72420605 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:54.421551 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a643da-583a-4df4-9380-4c6a72420605 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:54.422895 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a643da-583a-4df4-9380-4c6a72420605 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:54.425155 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 197/382] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:54.425981 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e1f6c-20f7-46bc-b70a-a7eee04ef4e7 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:54.445234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 186/383] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:54.449843 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0acba3-57cd-443a-9d4d-911bfd504b62 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:54.450563 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0acba3-57cd-443a-9d4d-911bfd504b62 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:54.450978 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0acba3-57cd-443a-9d4d-911bfd504b62 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:54.452715 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 187/384] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:54.453344 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0acba3-57cd-443a-9d4d-911bfd504b62 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:54.504965 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:54.531702 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e1f6c-20f7-46bc-b70a-a7eee04ef4e7 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:54.552050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:54.558825 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0acba3-57cd-443a-9d4d-911bfd504b62 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:54.565740 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b819dc72-eb69-4b2e-a44e-41ed31fc65be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:54.566442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819dc72-eb69-4b2e-a44e-41ed31fc65be None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:54.566786 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819dc72-eb69-4b2e-a44e-41ed31fc65be None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:54.567123 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819dc72-eb69-4b2e-a44e-41ed31fc65be None admin] PATH_INFO: `/v3/projects/service` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:54.574163 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b819dc72-eb69-4b2e-a44e-41ed31fc65be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:54.575580 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b819dc72-eb69-4b2e-a44e-41ed31fc65be None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:54.587253 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-b819dc72-eb69-4b2e-a44e-41ed31fc65be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:32:54.587253 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:54.587253 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:54.587253 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:54.587253 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:54.587253 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:54.587253 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:54.587253 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:54.587253 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:54.587253 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:54.587253 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:54.587253 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:54.587253 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:32:54.587253 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:32:54.587253 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:32:54.588153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:32:54.588153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:54.588153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:54.588153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:54.588153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:54.588153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:54.588153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:54.588153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:54.588153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:54.588153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:54.588153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:54.588153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:54.588153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:54.588153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:54.588153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:54.588153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:54.588153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:54.589021 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:32:54.589021 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:32:54.589021 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:32:54.589021 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:32:54.589021 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:32:54.589021 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:32:54.589021 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:32:54.589021 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:54.589021 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 198/385] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:32:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:54.589504 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b819dc72-eb69-4b2e-a44e-41ed31fc65be None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:54.598919 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b2f607-c313-4b82-8d0e-db4dad500e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:54.599544 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b2f607-c313-4b82-8d0e-db4dad500e29 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:54.599851 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b2f607-c313-4b82-8d0e-db4dad500e29 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:54.600123 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b2f607-c313-4b82-8d0e-db4dad500e29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:54.610637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b2f607-c313-4b82-8d0e-db4dad500e29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:54.612083 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b2f607-c313-4b82-8d0e-db4dad500e29 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:54.626804 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-56b2f607-c313-4b82-8d0e-db4dad500e29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:54.626804 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:54.626804 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:54.626804 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:54.626804 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:54.626804 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:54.626804 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:54.626804 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:54.626804 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:54.626804 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:54.626804 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:54.626804 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:54.626804 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:32:54.626804 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:32:54.626804 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:32:54.626804 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:32:54.627931 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:54.627931 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:54.627931 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:54.627931 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:54.627931 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:54.627931 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:54.627931 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:54.627931 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:54.627931 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:54.627931 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:54.627931 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:54.627931 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:54.627931 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:54.627931 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:54.627931 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:54.627931 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:54.627931 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:32:54.628948 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:32:54.628948 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:54.628948 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:54.634783 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 188/386] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:32:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:54.634783 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe59d2-9335-4884-9322-2a262b822c6e None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:54.634783 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe59d2-9335-4884-9322-2a262b822c6e None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:54.634783 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe59d2-9335-4884-9322-2a262b822c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:54.650684 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-d8fe59d2-9335-4884-9322-2a262b822c6e None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:54.652064 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-d8fe59d2-9335-4884-9322-2a262b822c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:54.700798 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b819dc72-eb69-4b2e-a44e-41ed31fc65be None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:54.722611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 189/387] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:54.723568 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe59d2-9335-4884-9322-2a262b822c6e None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:54.736212 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7775f0bb-4293-47d8-8a09-c63270347aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:54.736924 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775f0bb-4293-47d8-8a09-c63270347aed None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:54.737275 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775f0bb-4293-47d8-8a09-c63270347aed None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:54.737731 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775f0bb-4293-47d8-8a09-c63270347aed None admin] PATH_INFO: `/v3/users/glance` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:54.746389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7775f0bb-4293-47d8-8a09-c63270347aed None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:54.748051 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7775f0bb-4293-47d8-8a09-c63270347aed None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:54.754251 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:54.760935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-7775f0bb-4293-47d8-8a09-c63270347aed None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 17 03:32:54.760935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:54.760935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:54.760935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:54.760935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:54.760935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:54.760935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:54.760935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:54.760935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:54.760935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:54.760935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:54.760935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:54.760935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:32:54.760935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:32:54.760935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:32:54.760935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:32:54.762400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:54.762400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:54.762400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:32:54.762400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:32:54.762400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:32:54.762400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:32:54.762400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:54.762400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:54.762400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:54.762400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:54.762400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:54.762400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:54.762400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:54.762400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:54.762400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:54.762400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:54.762400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:54.762400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:54.763477 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:54.763477 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:54.763477 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:32:54.763477 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:32:54.763477 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:32:54.763477 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:32:54.763477 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:32:54.763477 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:32:54.763477 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 17 03:32:54.763477 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:54.763477 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 199/388] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:32:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:54.764618 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7775f0bb-4293-47d8-8a09-c63270347aed None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:54.809687 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:54.830086 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe59d2-9335-4884-9322-2a262b822c6e None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:54.866703 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a30883-652e-4dee-9d9e-122086f76f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:54.867390 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a30883-652e-4dee-9d9e-122086f76f5b None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:54.867759 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a30883-652e-4dee-9d9e-122086f76f5b None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:54.868106 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a30883-652e-4dee-9d9e-122086f76f5b None admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:54.868868 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a30883-652e-4dee-9d9e-122086f76f5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:54.869279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a30883-652e-4dee-9d9e-122086f76f5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:54.871294 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a30883-652e-4dee-9d9e-122086f76f5b None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:54.873728 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7775f0bb-4293-47d8-8a09-c63270347aed None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:54.893608 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 190/389] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Oct 17 03:32:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:54.893608 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a30883-652e-4dee-9d9e-122086f76f5b None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:54.912174 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef4d96-881f-46d4-ad75-5bc50e9c4561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:54.913090 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef4d96-881f-46d4-ad75-5bc50e9c4561 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:54.913370 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef4d96-881f-46d4-ad75-5bc50e9c4561 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:54.913639 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ef4d96-881f-46d4-ad75-5bc50e9c4561 None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:54.914271 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ef4d96-881f-46d4-ad75-5bc50e9c4561 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:54.914570 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ef4d96-881f-46d4-ad75-5bc50e9c4561 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:54.916387 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ef4d96-881f-46d4-ad75-5bc50e9c4561 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:54.928504 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 200/390] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:54.934548 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef4d96-881f-46d4-ad75-5bc50e9c4561 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:54.987985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:54.997404 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a30883-652e-4dee-9d9e-122086f76f5b None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:55.031013 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58879a-f1be-4c11-b0cb-1f013c7cb510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:55.031814 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58879a-f1be-4c11-b0cb-1f013c7cb510 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:55.032102 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58879a-f1be-4c11-b0cb-1f013c7cb510 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:55.032379 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58879a-f1be-4c11-b0cb-1f013c7cb510 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:55.033998 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:55.036877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ef4d96-881f-46d4-ad75-5bc50e9c4561 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:55.043228 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a58879a-f1be-4c11-b0cb-1f013c7cb510 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:55.048184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-1a58879a-f1be-4c11-b0cb-1f013c7cb510 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:32:55.048184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:55.048184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:55.048184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:55.048184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:55.048184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:55.048184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:55.048184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:55.048184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:55.048184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:55.048184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:55.048184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:55.048184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:55.048184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:55.048184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:55.048184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:55.049241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:55.049241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:55.049241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:55.049241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:55.049241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:55.049241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:55.049241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:55.049241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:55.049241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:55.049241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:55.049241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:55.049241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:55.049241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:55.049241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:55.049241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:55.049241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:55.050242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:55.050242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:55.050242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:55.050242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:55.050242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:55.050242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:55.050242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:55.050242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:55.050242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:55.050242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:55.050242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:32:55.050242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:55.050242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 191/391] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Oct 17 03:32:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:55.050242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58879a-f1be-4c11-b0cb-1f013c7cb510 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:55.075004 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635801e-d851-47d2-a5e4-31ad243306bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:55.075367 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635801e-d851-47d2-a5e4-31ad243306bb None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:55.075653 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635801e-d851-47d2-a5e4-31ad243306bb None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:55.075949 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635801e-d851-47d2-a5e4-31ad243306bb None admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:55.076673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635801e-d851-47d2-a5e4-31ad243306bb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:55.077079 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635801e-d851-47d2-a5e4-31ad243306bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:55.078392 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635801e-d851-47d2-a5e4-31ad243306bb None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:55.095597 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 201/392] 10.222.0.17 () {56 vars in 1137 bytes} [Mon Oct 17 03:32:54 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:55.096426 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635801e-d851-47d2-a5e4-31ad243306bb None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:55.148526 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58879a-f1be-4c11-b0cb-1f013c7cb510 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:55.182788 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251341a-70c0-463f-97a4-1fad82da8e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:55.183429 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251341a-70c0-463f-97a4-1fad82da8e13 None admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:55.183700 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251341a-70c0-463f-97a4-1fad82da8e13 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:55.184004 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251341a-70c0-463f-97a4-1fad82da8e13 None admin] PATH_INFO: `/v3/projects/932cdb9ef7644bf5ba57c6d1d6bce7ff/users/d961b751763a4e49b5d4ca4c795c5bf6/roles/8a0215180fe04d5a8f2c24b370382456` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:55.192337 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635801e-d851-47d2-a5e4-31ad243306bb None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:55.197622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b251341a-70c0-463f-97a4-1fad82da8e13 None admin] RBAC: Authorizing `identity:create_grant(project_id=932cdb9ef7644bf5ba57c6d1d6bce7ff, user_id=d961b751763a4e49b5d4ca4c795c5bf6, role_id=8a0215180fe04d5a8f2c24b370382456)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:55.199081 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b251341a-70c0-463f-97a4-1fad82da8e13 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:55.225618 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66613931-a4a5-43dd-acbc-8fcbbb17b8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:55.226237 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66613931-a4a5-43dd-acbc-8fcbbb17b8aa None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:55.226516 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66613931-a4a5-43dd-acbc-8fcbbb17b8aa None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:55.226790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66613931-a4a5-43dd-acbc-8fcbbb17b8aa None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:55.234174 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66613931-a4a5-43dd-acbc-8fcbbb17b8aa None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:55.235591 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66613931-a4a5-43dd-acbc-8fcbbb17b8aa None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:55.246904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-66613931-a4a5-43dd-acbc-8fcbbb17b8aa None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 03:32:55.246904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:55.246904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:55.246904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:55.246904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:55.246904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:55.246904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:55.246904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:55.246904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:55.246904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:55.246904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:55.246904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:55.246904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:32:55.246904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:32:55.246904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:32:55.246904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:32:55.247857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:55.247857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:55.247857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:32:55.247857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:32:55.247857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:32:55.247857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:32:55.247857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:55.247857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:55.247857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:55.247857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:55.247857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:55.247857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:55.247857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:55.247857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:55.247857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:55.247857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:55.247857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:55.247857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:55.248775 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:55.248775 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:55.248775 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:32:55.248775 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:32:55.248775 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:32:55.248775 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:32:55.248775 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:32:55.248775 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:32:55.248775 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 03:32:55.248775 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:55.248775 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 202/393] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:32:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:55.248775 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66613931-a4a5-43dd-acbc-8fcbbb17b8aa None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:55.266381 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:55.281772 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 192/394] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Oct 17 03:32:55 2022] PUT /identity/v3/projects/932cdb9ef7644bf5ba57c6d1d6bce7ff/users/d961b751763a4e49b5d4ca4c795c5bf6/roles/8a0215180fe04d5a8f2c24b370382456 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:32:55.282658 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251341a-70c0-463f-97a4-1fad82da8e13 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:55.285274 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:55.365930 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66613931-a4a5-43dd-acbc-8fcbbb17b8aa None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:55.403172 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e18cf9-3ebf-4d0e-b943-b8d74710aad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:55.403853 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e18cf9-3ebf-4d0e-b943-b8d74710aad3 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:55.404139 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e18cf9-3ebf-4d0e-b943-b8d74710aad3 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:55.404493 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e18cf9-3ebf-4d0e-b943-b8d74710aad3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:55.416149 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e18cf9-3ebf-4d0e-b943-b8d74710aad3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:55.418242 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e18cf9-3ebf-4d0e-b943-b8d74710aad3 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:55.422502 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251341a-70c0-463f-97a4-1fad82da8e13 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:55.436633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-e3e18cf9-3ebf-4d0e-b943-b8d74710aad3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:55.436633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:55.436633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:55.436633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:55.436633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:55.436633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:55.436633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:55.436633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:55.436633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:55.436633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:55.436633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:55.436633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:55.436633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:32:55.436633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:32:55.436633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:32:55.436633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:32:55.437606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:55.437606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:55.437606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:55.437606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:55.437606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:55.437606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:55.437606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:55.437606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:55.437606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:55.437606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:55.437606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:55.437606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:55.437606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:55.437606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:55.437606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:55.437606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:55.437606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:32:55.438424 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:32:55.438424 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:55.438424 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:55.438424 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 203/395] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:32:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:55.439155 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e18cf9-3ebf-4d0e-b943-b8d74710aad3 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:55.468186 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c7ede9-f950-4afc-8d59-413d88e02b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:55.468968 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7ede9-f950-4afc-8d59-413d88e02b63 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:55.469350 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7ede9-f950-4afc-8d59-413d88e02b63 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:55.469597 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7ede9-f950-4afc-8d59-413d88e02b63 None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:55.470179 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c7ede9-f950-4afc-8d59-413d88e02b63 None admin] RBAC: Adding query filter params (name=service) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:55.470507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c7ede9-f950-4afc-8d59-413d88e02b63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:55.471933 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c7ede9-f950-4afc-8d59-413d88e02b63 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:55.481802 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 193/396] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Oct 17 03:32:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:55.482864 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c7ede9-f950-4afc-8d59-413d88e02b63 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:55.519499 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:55.545545 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:55.553805 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e18cf9-3ebf-4d0e-b943-b8d74710aad3 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:55.583475 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c7ede9-f950-4afc-8d59-413d88e02b63 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:55.591815 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8394076-44e7-4dc9-8e30-f95361d0cbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:55.592481 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8394076-44e7-4dc9-8e30-f95361d0cbe7 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:55.592766 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8394076-44e7-4dc9-8e30-f95361d0cbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:55.593038 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8394076-44e7-4dc9-8e30-f95361d0cbe7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:55.604778 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8394076-44e7-4dc9-8e30-f95361d0cbe7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:55.606314 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8394076-44e7-4dc9-8e30-f95361d0cbe7 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:55.620643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-b8394076-44e7-4dc9-8e30-f95361d0cbe7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:55.620643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:55.620643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:55.620643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:55.620643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:55.620643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:55.620643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:55.620643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:55.620643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:55.620643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:55.620643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:55.620643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:55.620643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:32:55.620643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:32:55.620643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:32:55.620643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:32:55.621760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:55.621760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:55.621760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:55.621760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:55.621760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:55.621760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:55.621760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:55.621760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:55.621760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:55.621760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:55.621760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:55.621760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:55.621760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:55.621760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:55.621760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:55.621760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:55.621760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:32:55.622782 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:32:55.622782 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:55.622782 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:55.625613 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7084be-d544-4c58-b764-beae4d98db99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:55.626506 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7084be-d544-4c58-b764-beae4d98db99 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:55.626606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 204/397] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:32:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:55.626675 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7084be-d544-4c58-b764-beae4d98db99 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:55.626874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7084be-d544-4c58-b764-beae4d98db99 None admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:55.627124 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8394076-44e7-4dc9-8e30-f95361d0cbe7 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:55.627507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7084be-d544-4c58-b764-beae4d98db99 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:55.627894 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7084be-d544-4c58-b764-beae4d98db99 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:55.629248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7084be-d544-4c58-b764-beae4d98db99 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:55.669874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 194/398] 10.222.0.17 () {56 vars in 1137 bytes} [Mon Oct 17 03:32:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:55.670717 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7084be-d544-4c58-b764-beae4d98db99 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:55.748822 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8394076-44e7-4dc9-8e30-f95361d0cbe7 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:55.772165 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:55.781275 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7084be-d544-4c58-b764-beae4d98db99 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:55.784680 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ae041-da0f-4afa-854c-a29ad9358240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:55.785966 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ae041-da0f-4afa-854c-a29ad9358240 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:55.785966 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ae041-da0f-4afa-854c-a29ad9358240 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:55.786445 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721ae041-da0f-4afa-854c-a29ad9358240 None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:55.787229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ae041-da0f-4afa-854c-a29ad9358240 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:55.787685 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ae041-da0f-4afa-854c-a29ad9358240 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:55.789241 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721ae041-da0f-4afa-854c-a29ad9358240 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:55.801743 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 205/399] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:55.804641 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ae041-da0f-4afa-854c-a29ad9358240 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:55.809427 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:55.821263 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e561e2c6-3448-4f8d-99e5-2b08e39a0b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:55.825727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e561e2c6-3448-4f8d-99e5-2b08e39a0b31 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:55.825727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e561e2c6-3448-4f8d-99e5-2b08e39a0b31 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:55.825727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e561e2c6-3448-4f8d-99e5-2b08e39a0b31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:55.833294 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e561e2c6-3448-4f8d-99e5-2b08e39a0b31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:55.834830 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e561e2c6-3448-4f8d-99e5-2b08e39a0b31 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:55.851883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-e561e2c6-3448-4f8d-99e5-2b08e39a0b31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:55.851883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:55.851883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:55.851883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:55.851883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:55.851883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:55.851883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:55.851883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:55.851883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:55.851883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:55.851883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:55.851883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:55.851883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:32:55.851883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:32:55.851883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:32:55.851883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:32:55.853005 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:55.853005 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:55.853005 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:55.853005 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:55.853005 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:55.853005 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:55.853005 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:55.853005 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:55.853005 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:55.853005 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:55.853005 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:55.853005 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:55.853005 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:55.853005 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:55.853005 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:55.853005 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:55.853005 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:32:55.854103 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:32:55.854103 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:55.854103 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:55.855875 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 195/400] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:32:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:55.856684 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e561e2c6-3448-4f8d-99e5-2b08e39a0b31 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:55.924947 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721ae041-da0f-4afa-854c-a29ad9358240 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:55.961789 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454a6b9-dd68-44c2-98b2-a3de2dfca266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:55.962403 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454a6b9-dd68-44c2-98b2-a3de2dfca266 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:55.962682 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454a6b9-dd68-44c2-98b2-a3de2dfca266 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:55.962953 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454a6b9-dd68-44c2-98b2-a3de2dfca266 None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:55.963194 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e561e2c6-3448-4f8d-99e5-2b08e39a0b31 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:55.963617 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454a6b9-dd68-44c2-98b2-a3de2dfca266 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:55.963938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454a6b9-dd68-44c2-98b2-a3de2dfca266 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:55.965593 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a454a6b9-dd68-44c2-98b2-a3de2dfca266 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:55.981504 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 206/401] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:55.981504 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454a6b9-dd68-44c2-98b2-a3de2dfca266 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:56.004561 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380c1b6-f3a3-4b5e-86ad-fed98b93a5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:56.005112 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380c1b6-f3a3-4b5e-86ad-fed98b93a5f2 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:56.005397 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380c1b6-f3a3-4b5e-86ad-fed98b93a5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:56.005673 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380c1b6-f3a3-4b5e-86ad-fed98b93a5f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:56.016190 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3380c1b6-f3a3-4b5e-86ad-fed98b93a5f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:56.017635 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3380c1b6-f3a3-4b5e-86ad-fed98b93a5f2 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:56.026691 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:56.032159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-3380c1b6-f3a3-4b5e-86ad-fed98b93a5f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:56.032159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:56.032159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:56.032159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:56.032159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:56.032159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:56.032159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:56.032159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:56.032159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:56.032159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:56.032159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:56.032159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:56.032159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:32:56.032159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:32:56.032159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:32:56.032159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:32:56.038642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:56.038642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:56.038642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:56.038642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:56.038642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:56.038642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:56.038642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:56.038642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:56.038642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:56.038642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:56.038642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:56.038642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:56.038642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:56.038642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:56.038642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:56.038642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:56.038642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:32:56.039695 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:32:56.039695 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:56.039695 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:56.039695 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 196/402] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:32:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:56.039695 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380c1b6-f3a3-4b5e-86ad-fed98b93a5f2 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:56.064746 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:56.099600 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454a6b9-dd68-44c2-98b2-a3de2dfca266 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:56.136246 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7f995-d970-426d-a245-2bf9b67a11e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:56.137020 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7f995-d970-426d-a245-2bf9b67a11e7 None admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:56.137370 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7f995-d970-426d-a245-2bf9b67a11e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:56.137717 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd7f995-d970-426d-a245-2bf9b67a11e7 None admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:56.138433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7f995-d970-426d-a245-2bf9b67a11e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:56.140047 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd7f995-d970-426d-a245-2bf9b67a11e7 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:56.143759 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380c1b6-f3a3-4b5e-86ad-fed98b93a5f2 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:56.179632 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14dc227-5f57-44ab-b909-6d8926f03373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:56.180319 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14dc227-5f57-44ab-b909-6d8926f03373 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:56.180535 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14dc227-5f57-44ab-b909-6d8926f03373 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:56.180806 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14dc227-5f57-44ab-b909-6d8926f03373 None admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:56.181449 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14dc227-5f57-44ab-b909-6d8926f03373 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:56.181751 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14dc227-5f57-44ab-b909-6d8926f03373 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:56.183784 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14dc227-5f57-44ab-b909-6d8926f03373 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:56.194960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 197/403] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:56.196687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14dc227-5f57-44ab-b909-6d8926f03373 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:56.218568 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 207/404] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Oct 17 03:32:55 2022] POST /identity/v3/users => generated 247 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:32:56.229690 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7f995-d970-426d-a245-2bf9b67a11e7 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:56.311387 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14dc227-5f57-44ab-b909-6d8926f03373 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:56.320238 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:56.350081 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672fc12-32aa-40ab-94da-f3f94e9c19bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:56.350857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672fc12-32aa-40ab-94da-f3f94e9c19bc None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:56.351145 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672fc12-32aa-40ab-94da-f3f94e9c19bc None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:56.351422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672fc12-32aa-40ab-94da-f3f94e9c19bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:56.360106 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672fc12-32aa-40ab-94da-f3f94e9c19bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:56.362086 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672fc12-32aa-40ab-94da-f3f94e9c19bc None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:56.366557 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:56.371183 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd7f995-d970-426d-a245-2bf9b67a11e7 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:56.377351 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-d672fc12-32aa-40ab-94da-f3f94e9c19bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:32:56.377351 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:56.377351 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:56.377351 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:56.377351 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:56.377351 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:56.377351 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:56.377351 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:56.377351 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:56.377351 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:56.377351 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:56.377351 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:56.377351 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:32:56.377351 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:32:56.377351 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:32:56.381490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:32:56.381490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:56.381490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:56.381490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:56.381490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:56.381490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:56.381490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:56.381490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:56.381490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:56.381490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:56.381490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:56.381490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:56.381490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:56.381490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:56.381490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:56.381490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:56.381490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:56.382365 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:32:56.382365 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:32:56.382365 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:32:56.382365 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:32:56.382365 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:32:56.382365 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:32:56.382365 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:32:56.382365 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:56.382365 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 198/405] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:32:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:56.382365 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672fc12-32aa-40ab-94da-f3f94e9c19bc None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:56.416883 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3bda6f-fd4a-4802-9d92-b22a25c3001c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:56.417511 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3bda6f-fd4a-4802-9d92-b22a25c3001c None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:56.417795 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3bda6f-fd4a-4802-9d92-b22a25c3001c None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:56.418072 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3bda6f-fd4a-4802-9d92-b22a25c3001c None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:56.418722 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3bda6f-fd4a-4802-9d92-b22a25c3001c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:56.419026 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3bda6f-fd4a-4802-9d92-b22a25c3001c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:56.420492 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3bda6f-fd4a-4802-9d92-b22a25c3001c None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:56.434816 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 208/406] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:56.435966 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3bda6f-fd4a-4802-9d92-b22a25c3001c None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:56.509957 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672fc12-32aa-40ab-94da-f3f94e9c19bc None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:56.547259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8c060-4379-4ec8-bf66-79aa6103c925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:56.548277 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8c060-4379-4ec8-bf66-79aa6103c925 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:56.548730 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8c060-4379-4ec8-bf66-79aa6103c925 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:56.549168 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8c060-4379-4ec8-bf66-79aa6103c925 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:56.558028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb8c060-4379-4ec8-bf66-79aa6103c925 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:56.559869 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb8c060-4379-4ec8-bf66-79aa6103c925 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:56.561802 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3bda6f-fd4a-4802-9d92-b22a25c3001c None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:56.574139 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-2fb8c060-4379-4ec8-bf66-79aa6103c925 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 17 03:32:56.574139 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:56.574139 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:56.574139 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:56.574139 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:56.574139 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:56.574139 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:56.574139 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:56.574139 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:56.574139 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:56.574139 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:56.574139 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:56.574139 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:32:56.574139 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:32:56.574139 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:32:56.574139 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:32:56.575054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:56.575054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:56.575054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:32:56.575054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:32:56.575054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:32:56.575054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:32:56.575054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:56.575054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:56.575054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:56.575054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:56.575054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:56.575054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:56.575054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:56.575054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:56.575054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:56.575054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:56.575054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:56.575054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:56.575985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:56.575985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:56.575985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:32:56.575985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:32:56.575985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:32:56.575985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:32:56.575985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:32:56.575985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:32:56.575985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 17 03:32:56.575985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:56.577890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 199/407] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Oct 17 03:32:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:56.578973 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8c060-4379-4ec8-bf66-79aa6103c925 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:56.590227 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:56.601661 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f55c8-e6be-419f-a498-ff2949d701fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:56.602573 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f55c8-e6be-419f-a498-ff2949d701fe None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:56.603980 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f55c8-e6be-419f-a498-ff2949d701fe None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:56.603980 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f55c8-e6be-419f-a498-ff2949d701fe None admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:56.603980 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f55c8-e6be-419f-a498-ff2949d701fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:56.604732 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f55c8-e6be-419f-a498-ff2949d701fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:56.606179 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f55c8-e6be-419f-a498-ff2949d701fe None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:56.625878 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 209/408] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Oct 17 03:32:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:56.626908 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f55c8-e6be-419f-a498-ff2949d701fe None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:56.646631 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:56.700425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8c060-4379-4ec8-bf66-79aa6103c925 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:56.739934 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f55c8-e6be-419f-a498-ff2949d701fe None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:56.740159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec21377-5428-4c33-866e-52e0828351b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:56.740660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec21377-5428-4c33-866e-52e0828351b4 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:56.740939 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec21377-5428-4c33-866e-52e0828351b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:56.741218 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec21377-5428-4c33-866e-52e0828351b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:56.748776 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec21377-5428-4c33-866e-52e0828351b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:56.750248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec21377-5428-4c33-866e-52e0828351b4 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:56.762803 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-3ec21377-5428-4c33-866e-52e0828351b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:32:56.762803 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:56.762803 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:56.762803 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:56.762803 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:56.762803 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:56.762803 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:56.762803 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:56.762803 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:56.762803 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:56.762803 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:56.762803 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:56.762803 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:32:56.762803 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:32:56.762803 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:32:56.763874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:32:56.763874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:56.763874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:56.763874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:56.763874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:56.763874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:56.763874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:56.763874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:56.763874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:56.763874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:56.763874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:56.763874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:56.763874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:56.763874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:56.763874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:56.763874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:56.763874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:56.764950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:32:56.764950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:32:56.764950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:32:56.764950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:32:56.764950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:32:56.764950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:32:56.764950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:32:56.764950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:56.764950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 200/409] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:32:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:56.764950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec21377-5428-4c33-866e-52e0828351b4 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:56.779383 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b4f13-a9d8-45ff-ada6-36d3251dc49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:56.780255 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b4f13-a9d8-45ff-ada6-36d3251dc49a None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:56.780706 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b4f13-a9d8-45ff-ada6-36d3251dc49a None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:56.781133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b4f13-a9d8-45ff-ada6-36d3251dc49a None admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:56.782014 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515b4f13-a9d8-45ff-ada6-36d3251dc49a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:56.782530 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515b4f13-a9d8-45ff-ada6-36d3251dc49a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:56.784130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515b4f13-a9d8-45ff-ada6-36d3251dc49a None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:56.802474 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 210/410] 10.222.0.17 () {56 vars in 1139 bytes} [Mon Oct 17 03:32:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:56.803521 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b4f13-a9d8-45ff-ada6-36d3251dc49a None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:56.863946 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:56.873940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec21377-5428-4c33-866e-52e0828351b4 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:56.883066 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:56.906981 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b4f13-a9d8-45ff-ada6-36d3251dc49a None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:56.911727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a375b-e4e4-477e-b067-dff5ec22b01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:56.912653 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a375b-e4e4-477e-b067-dff5ec22b01a None admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:56.913122 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a375b-e4e4-477e-b067-dff5ec22b01a None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:56.913540 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a375b-e4e4-477e-b067-dff5ec22b01a None admin] PATH_INFO: `/v3/projects/932cdb9ef7644bf5ba57c6d1d6bce7ff/users/565a836c7f89478cad615bd6c14531a7/roles/8a0215180fe04d5a8f2c24b370382456` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:56.929730 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a375b-e4e4-477e-b067-dff5ec22b01a None admin] RBAC: Authorizing `identity:create_grant(project_id=932cdb9ef7644bf5ba57c6d1d6bce7ff, user_id=565a836c7f89478cad615bd6c14531a7, role_id=8a0215180fe04d5a8f2c24b370382456)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:56.932563 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a375b-e4e4-477e-b067-dff5ec22b01a None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:56.942621 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bcb377-3f41-48ab-845c-34e9f5ecca09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:56.943232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bcb377-3f41-48ab-845c-34e9f5ecca09 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:56.943510 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bcb377-3f41-48ab-845c-34e9f5ecca09 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:56.949050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bcb377-3f41-48ab-845c-34e9f5ecca09 None admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:56.949050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bcb377-3f41-48ab-845c-34e9f5ecca09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:56.949050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bcb377-3f41-48ab-845c-34e9f5ecca09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:56.949050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bcb377-3f41-48ab-845c-34e9f5ecca09 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:56.958451 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 211/411] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Oct 17 03:32:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:56.959435 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bcb377-3f41-48ab-845c-34e9f5ecca09 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:57.003404 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 201/412] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Oct 17 03:32:56 2022] PUT /identity/v3/projects/932cdb9ef7644bf5ba57c6d1d6bce7ff/users/565a836c7f89478cad615bd6c14531a7/roles/8a0215180fe04d5a8f2c24b370382456 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:32:57.004284 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a375b-e4e4-477e-b067-dff5ec22b01a None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:57.139704 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a375b-e4e4-477e-b067-dff5ec22b01a None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:57.160032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bcb377-3f41-48ab-845c-34e9f5ecca09 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:57.167589 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:57.184425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160cef1-cfc0-4956-b306-ced5f35da6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:57.185369 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160cef1-cfc0-4956-b306-ced5f35da6ca None admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:57.185665 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160cef1-cfc0-4956-b306-ced5f35da6ca None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:57.185945 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160cef1-cfc0-4956-b306-ced5f35da6ca None admin] PATH_INFO: `/v3/projects/932cdb9ef7644bf5ba57c6d1d6bce7ff/users/99bc94105bbf4cb4a60e37133edc8dfb/roles/8a0215180fe04d5a8f2c24b370382456` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:57.207170 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3160cef1-cfc0-4956-b306-ced5f35da6ca None admin] RBAC: Authorizing `identity:create_grant(project_id=932cdb9ef7644bf5ba57c6d1d6bce7ff, user_id=99bc94105bbf4cb4a60e37133edc8dfb, role_id=8a0215180fe04d5a8f2c24b370382456)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:57.209080 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3160cef1-cfc0-4956-b306-ced5f35da6ca None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:57.214773 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b2d1c-f978-453f-bc48-95ddf740032f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:57.215723 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b2d1c-f978-453f-bc48-95ddf740032f None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:57.216078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b2d1c-f978-453f-bc48-95ddf740032f None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:57.218243 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6b2d1c-f978-453f-bc48-95ddf740032f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:57.220026 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:57.229905 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b2d1c-f978-453f-bc48-95ddf740032f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:57.231448 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6b2d1c-f978-453f-bc48-95ddf740032f None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:57.249452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-ce6b2d1c-f978-453f-bc48-95ddf740032f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:57.249452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:57.249452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:57.249452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:57.249452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:57.249452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:57.249452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:57.249452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:57.249452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:57.249452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:57.249452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:57.249452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:57.249452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:32:57.249452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:32:57.249452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:32:57.249452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:32:57.250591 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:57.250591 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:57.250591 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:57.250591 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:57.250591 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:57.250591 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:57.250591 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:57.250591 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:57.250591 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:57.250591 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:57.250591 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:57.250591 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:57.250591 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:57.250591 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:57.250591 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:57.250591 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:57.250591 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:32:57.251683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:32:57.251683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:57.251683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:57.253018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 212/413] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:32:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:57.258434 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b2d1c-f978-453f-bc48-95ddf740032f None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:57.302147 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 202/414] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Oct 17 03:32:57 2022] PUT /identity/v3/projects/932cdb9ef7644bf5ba57c6d1d6bce7ff/users/99bc94105bbf4cb4a60e37133edc8dfb/roles/8a0215180fe04d5a8f2c24b370382456 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:32:57.401299 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6b2d1c-f978-453f-bc48-95ddf740032f None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:57.451682 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c3487-0d18-4f95-aa14-c9a5c58376f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:57.452339 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c3487-0d18-4f95-aa14-c9a5c58376f7 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:57.452618 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c3487-0d18-4f95-aa14-c9a5c58376f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:57.452888 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c3487-0d18-4f95-aa14-c9a5c58376f7 None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:57.453538 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0c3487-0d18-4f95-aa14-c9a5c58376f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:57.453838 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0c3487-0d18-4f95-aa14-c9a5c58376f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:57.455206 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0c3487-0d18-4f95-aa14-c9a5c58376f7 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:57.468927 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4c8d9-d7d2-430d-afed-9d8d533e9a3c None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:57.469543 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4c8d9-d7d2-430d-afed-9d8d533e9a3c None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:57.470059 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4c8d9-d7d2-430d-afed-9d8d533e9a3c None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:57.471680 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 203/415] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:32:57.476747 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 213/416] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:57.481334 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6f3dd-cab1-4c29-a4e7-5f0491330404 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:57.481911 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6f3dd-cab1-4c29-a4e7-5f0491330404 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:57.482041 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c3487-0d18-4f95-aa14-c9a5c58376f7 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:57.482549 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6f3dd-cab1-4c29-a4e7-5f0491330404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:57.518011 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-82c6f3dd-cab1-4c29-a4e7-5f0491330404 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:57.519947 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-82c6f3dd-cab1-4c29-a4e7-5f0491330404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:57.565277 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:57.618222 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 204/417] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:32:57.626648 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0c3487-0d18-4f95-aa14-c9a5c58376f7 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:57.663036 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1700e1-3f06-4a12-9337-342fc723a99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:57.664116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1700e1-3f06-4a12-9337-342fc723a99d None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:57.664535 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1700e1-3f06-4a12-9337-342fc723a99d None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:57.664945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1700e1-3f06-4a12-9337-342fc723a99d None admin] PATH_INFO: `/v3/projects/service` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:57.672106 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1700e1-3f06-4a12-9337-342fc723a99d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:57.673718 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1700e1-3f06-4a12-9337-342fc723a99d None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:57.686235 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-ea1700e1-3f06-4a12-9337-342fc723a99d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:32:57.686235 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:57.686235 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:57.686235 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:57.686235 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:57.686235 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:57.686235 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:57.686235 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:57.686235 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:57.686235 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:57.686235 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:57.686235 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:57.686235 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:32:57.686235 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:32:57.686235 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:32:57.687052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:32:57.687052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:57.687052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:57.687052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:57.687052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:57.687052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:57.687052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:57.687052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:57.687052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:57.687052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:57.687052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:57.687052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:57.687052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:57.687052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:57.687052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:57.687052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:57.687052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:57.687917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:32:57.687917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:32:57.687917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:32:57.687917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:32:57.687917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:32:57.687917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:32:57.687917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:32:57.687917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:57.687917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 214/418] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:32:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:32:57.694711 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6f3dd-cab1-4c29-a4e7-5f0491330404 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:57.795625 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:57.803903 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6f3dd-cab1-4c29-a4e7-5f0491330404 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:57.840562 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df4244-599d-47f3-950b-357649fd226e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:57.841528 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df4244-599d-47f3-950b-357649fd226e None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:57.842006 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df4244-599d-47f3-950b-357649fd226e None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:57.842472 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df4244-599d-47f3-950b-357649fd226e None admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:57.843362 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df4244-599d-47f3-950b-357649fd226e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:57.843848 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df4244-599d-47f3-950b-357649fd226e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:57.845560 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df4244-599d-47f3-950b-357649fd226e None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:57.858883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 205/419] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Oct 17 03:32:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:57.864904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1700e1-3f06-4a12-9337-342fc723a99d None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:58.002824 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1700e1-3f06-4a12-9337-342fc723a99d None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:58.011895 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:58.012361 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df4244-599d-47f3-950b-357649fd226e None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:58.060055 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c9b415-6e38-4ad2-96ee-eb6af19915f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:58.060726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c9b415-6e38-4ad2-96ee-eb6af19915f4 None admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:58.061005 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c9b415-6e38-4ad2-96ee-eb6af19915f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:58.061277 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c9b415-6e38-4ad2-96ee-eb6af19915f4 None admin] PATH_INFO: `/v3/projects/932cdb9ef7644bf5ba57c6d1d6bce7ff/users/414707c545f641d3935720356b326672/roles/8a0215180fe04d5a8f2c24b370382456` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:58.087056 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c9b415-6e38-4ad2-96ee-eb6af19915f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=932cdb9ef7644bf5ba57c6d1d6bce7ff, user_id=414707c545f641d3935720356b326672, role_id=8a0215180fe04d5a8f2c24b370382456)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:58.090373 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c9b415-6e38-4ad2-96ee-eb6af19915f4 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:58.136309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df4244-599d-47f3-950b-357649fd226e None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:58.172394 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e20d1-7dc7-42ea-beb6-baea272ab866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:58.173030 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e20d1-7dc7-42ea-beb6-baea272ab866 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:58.173306 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e20d1-7dc7-42ea-beb6-baea272ab866 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:58.173572 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e20d1-7dc7-42ea-beb6-baea272ab866 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:58.182401 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 215/420] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Oct 17 03:32:57 2022] PUT /identity/v3/projects/932cdb9ef7644bf5ba57c6d1d6bce7ff/users/414707c545f641d3935720356b326672/roles/8a0215180fe04d5a8f2c24b370382456 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:32:58.187585 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e20d1-7dc7-42ea-beb6-baea272ab866 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:58.193496 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-1f7e20d1-7dc7-42ea-beb6-baea272ab866 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:32:58.193496 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:58.193496 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:58.193496 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:58.193496 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:58.193496 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:58.193496 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:58.193496 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:58.193496 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:58.193496 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:58.193496 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:58.193496 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:58.193496 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:58.193496 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:58.193496 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:58.193496 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:58.194535 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:58.194535 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:58.194535 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:58.194535 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:58.194535 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:58.194535 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:58.194535 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:58.194535 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:58.194535 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:58.194535 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:58.194535 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:58.194535 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:58.194535 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:58.194535 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:58.194535 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:58.194535 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:58.195979 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:58.195979 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:58.195979 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:58.195979 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:58.195979 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:58.195979 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:58.195979 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:58.195979 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:58.195979 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:58.195979 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:58.195979 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:32:58.195979 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:58.195979 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 206/421] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Oct 17 03:32:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:58.200857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c9b415-6e38-4ad2-96ee-eb6af19915f4 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:58.371210 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c9b415-6e38-4ad2-96ee-eb6af19915f4 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:58.410316 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f028318-8e81-4bae-b55b-5a95c06000e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:58.410934 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f028318-8e81-4bae-b55b-5a95c06000e3 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:58.411208 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f028318-8e81-4bae-b55b-5a95c06000e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:58.411473 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f028318-8e81-4bae-b55b-5a95c06000e3 None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:58.412160 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f028318-8e81-4bae-b55b-5a95c06000e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:58.412459 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f028318-8e81-4bae-b55b-5a95c06000e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:58.413815 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f028318-8e81-4bae-b55b-5a95c06000e3 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:58.423479 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 216/422] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Oct 17 03:32:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:58.429240 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e20d1-7dc7-42ea-beb6-baea272ab866 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:58.453906 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:58.550191 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e20d1-7dc7-42ea-beb6-baea272ab866 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:58.587636 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bd349-18ef-4d64-ba89-d1a00e6e2e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:58.588956 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bd349-18ef-4d64-ba89-d1a00e6e2e67 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:58.589235 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bd349-18ef-4d64-ba89-d1a00e6e2e67 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:58.589506 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bd349-18ef-4d64-ba89-d1a00e6e2e67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:58.600016 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3bd349-18ef-4d64-ba89-d1a00e6e2e67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:58.601450 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3bd349-18ef-4d64-ba89-d1a00e6e2e67 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:58.616166 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-eb3bd349-18ef-4d64-ba89-d1a00e6e2e67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:58.616166 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:58.616166 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:58.616166 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:58.616166 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:58.616166 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:58.616166 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:58.616166 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:58.616166 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:58.616166 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:58.616166 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:58.616166 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:58.616166 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:32:58.616166 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:32:58.616166 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:32:58.616166 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:32:58.617038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:58.617038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:58.617038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:58.617038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:58.617038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:58.617038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:58.617038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:58.617038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:58.617038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:58.617038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:58.617038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:58.617038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:58.617038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:58.617038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:58.617038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:58.617038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:58.617038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:32:58.618263 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:32:58.618263 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:58.618263 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:58.618263 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 207/423] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:32:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:58.623123 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f028318-8e81-4bae-b55b-5a95c06000e3 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:58.646793 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:58.749055 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f028318-8e81-4bae-b55b-5a95c06000e3 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:58.789096 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df43f66-066c-48ff-9f5a-53b322949451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:58.790338 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df43f66-066c-48ff-9f5a-53b322949451 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:58.790848 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df43f66-066c-48ff-9f5a-53b322949451 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:58.791360 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df43f66-066c-48ff-9f5a-53b322949451 None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:58.792263 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df43f66-066c-48ff-9f5a-53b322949451 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:58.792791 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df43f66-066c-48ff-9f5a-53b322949451 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:58.794392 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df43f66-066c-48ff-9f5a-53b322949451 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:58.807327 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 217/424] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:32:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:58.812815 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bd349-18ef-4d64-ba89-d1a00e6e2e67 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:58.897563 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af549347-e7de-45ca-9094-0a49a1c57c00 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:58.898174 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af549347-e7de-45ca-9094-0a49a1c57c00 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:58.898639 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af549347-e7de-45ca-9094-0a49a1c57c00 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:58.900605 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 218/425] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:58.905638 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb46b68-eafd-4ac3-9467-c87948dfa5b2 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:58.906130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb46b68-eafd-4ac3-9467-c87948dfa5b2 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:58.906633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb46b68-eafd-4ac3-9467-c87948dfa5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:58.931238 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:58.933389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-afb46b68-eafd-4ac3-9467-c87948dfa5b2 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:32:58.934595 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-afb46b68-eafd-4ac3-9467-c87948dfa5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:32:58.969258 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bd349-18ef-4d64-ba89-d1a00e6e2e67 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:59.032600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ceec28-5d53-4721-8348-ebec0825d4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:59.033239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ceec28-5d53-4721-8348-ebec0825d4ee None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:59.033512 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ceec28-5d53-4721-8348-ebec0825d4ee None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:59.033778 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ceec28-5d53-4721-8348-ebec0825d4ee None admin] PATH_INFO: `/v3/users/nova` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:59.044889 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ceec28-5d53-4721-8348-ebec0825d4ee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:59.046332 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ceec28-5d53-4721-8348-ebec0825d4ee None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:59.066814 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-85ceec28-5d53-4721-8348-ebec0825d4ee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 03:32:59.066814 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:59.066814 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:59.066814 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:59.066814 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:59.066814 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:59.066814 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:59.066814 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:59.066814 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:59.066814 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:59.066814 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:59.066814 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:59.066814 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:32:59.066814 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:32:59.066814 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:32:59.066814 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:32:59.067716 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:59.067716 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:59.067716 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:32:59.067716 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:32:59.067716 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:32:59.067716 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:32:59.067716 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:59.067716 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:59.067716 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:59.067716 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:59.067716 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:59.067716 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:59.067716 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:59.067716 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:59.067716 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:59.067716 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:59.067716 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:59.067716 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:59.068649 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:59.068649 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:59.068649 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:32:59.068649 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:32:59.068649 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:32:59.068649 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:32:59.068649 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:32:59.068649 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:32:59.068649 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 03:32:59.068649 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:59.068649 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 208/426] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Oct 17 03:32:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:59.082486 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ceec28-5d53-4721-8348-ebec0825d4ee None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:59.084264 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 219/427] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:32:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 03:32:59.194657 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ceec28-5d53-4721-8348-ebec0825d4ee None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:59.222464 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:59.231085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6b0b3-145b-4bf9-ad56-1046b0c7cb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:59.231702 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6b0b3-145b-4bf9-ad56-1046b0c7cb28 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:59.232014 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6b0b3-145b-4bf9-ad56-1046b0c7cb28 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:59.242149 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df6b0b3-145b-4bf9-ad56-1046b0c7cb28 None admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:59.242149 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df6b0b3-145b-4bf9-ad56-1046b0c7cb28 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:32:59.242149 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df6b0b3-145b-4bf9-ad56-1046b0c7cb28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:59.242149 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df6b0b3-145b-4bf9-ad56-1046b0c7cb28 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:59.253677 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 209/428] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Oct 17 03:32:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:32:59.258810 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb46b68-eafd-4ac3-9467-c87948dfa5b2 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:59.283481 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:59.400916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb46b68-eafd-4ac3-9467-c87948dfa5b2 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:59.450343 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d112f6-2a9e-4a4f-91a5-7e0bbd88b83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:59.451314 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d112f6-2a9e-4a4f-91a5-7e0bbd88b83b None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:59.451841 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d112f6-2a9e-4a4f-91a5-7e0bbd88b83b None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:59.452310 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d112f6-2a9e-4a4f-91a5-7e0bbd88b83b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:59.532564 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6b0b3-145b-4bf9-ad56-1046b0c7cb28 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:59.540644 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d112f6-2a9e-4a4f-91a5-7e0bbd88b83b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:59.542917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d112f6-2a9e-4a4f-91a5-7e0bbd88b83b None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:59.560444 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:59.693697 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 220/429] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:32:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 03:32:59.700868 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d112f6-2a9e-4a4f-91a5-7e0bbd88b83b None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:59.733748 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df6b0b3-145b-4bf9-ad56-1046b0c7cb28 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:59.807911 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145bf78-a412-44d0-bd7a-4b2a60ebb30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:59.808553 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145bf78-a412-44d0-bd7a-4b2a60ebb30e None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:59.808831 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145bf78-a412-44d0-bd7a-4b2a60ebb30e None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:59.809104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145bf78-a412-44d0-bd7a-4b2a60ebb30e None admin] PATH_INFO: `/v3/roles/service` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:59.840887 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145bf78-a412-44d0-bd7a-4b2a60ebb30e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:32:59.844812 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d112f6-2a9e-4a4f-91a5-7e0bbd88b83b None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:32:59.846364 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-7145bf78-a412-44d0-bd7a-4b2a60ebb30e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:32:59.846364 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:59.846364 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:59.846364 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:59.846364 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:59.846364 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:59.846364 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:59.846364 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:59.846364 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:59.846364 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:59.846364 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:59.846364 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:59.846364 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:32:59.846364 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:32:59.846364 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:32:59.846364 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:32:59.847234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:32:59.847234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:32:59.847234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:32:59.847234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:32:59.847234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:59.847234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:59.847234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:59.847234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:59.847234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:59.847234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:59.847234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:59.847234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:59.847234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:59.847234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:59.847234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:59.847234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:59.848070 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:59.848070 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:59.848070 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:59.848070 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:59.848070 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:32:59.848070 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:32:59.848070 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:32:59.848070 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:32:59.848070 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:32:59.848070 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:32:59.848070 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:32:59.848070 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:32:59.856144 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:59.857629 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 210/430] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Oct 17 03:32:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:59.864457 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145bf78-a412-44d0-bd7a-4b2a60ebb30e None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:59.889474 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07376228-1e31-4b32-b2a9-1380c3b6f35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:32:59.890403 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07376228-1e31-4b32-b2a9-1380c3b6f35a None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:59.890880 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07376228-1e31-4b32-b2a9-1380c3b6f35a None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:59.891592 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07376228-1e31-4b32-b2a9-1380c3b6f35a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:59.905044 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07376228-1e31-4b32-b2a9-1380c3b6f35a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:32:59.906492 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07376228-1e31-4b32-b2a9-1380c3b6f35a None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:32:59.921557 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-07376228-1e31-4b32-b2a9-1380c3b6f35a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:59.921557 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:32:59.921557 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:32:59.921557 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:32:59.921557 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:32:59.921557 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:32:59.921557 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:32:59.921557 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:32:59.921557 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:32:59.921557 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:32:59.921557 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:32:59.921557 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:32:59.921557 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:32:59.921557 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:32:59.921557 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:32:59.921557 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:32:59.922457 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:32:59.922457 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:32:59.922457 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:32:59.922457 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:32:59.922457 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:32:59.922457 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:32:59.922457 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:32:59.922457 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:32:59.922457 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:32:59.922457 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:32:59.922457 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:32:59.922457 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:32:59.922457 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:32:59.922457 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:32:59.922457 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:32:59.922457 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:32:59.922457 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:32:59.923269 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:32:59.923269 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:32:59.923269 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:32:59.923269 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 221/431] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:32:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:32:59.923964 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b172e6d-b91a-4fcc-a14a-ae62191a3ee6 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:59.924243 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b172e6d-b91a-4fcc-a14a-ae62191a3ee6 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:59.924510 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b172e6d-b91a-4fcc-a14a-ae62191a3ee6 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:59.925716 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 222/432] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:59.929538 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:32:59.930405 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce6c37-19ed-490f-bd91-fa8fe7fe8100 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:32:59.930688 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce6c37-19ed-490f-bd91-fa8fe7fe8100 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:32:59.930956 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ce6c37-19ed-490f-bd91-fa8fe7fe8100 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:32:59.932305 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 223/433] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:32:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:32:59.933001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce6c37-19ed-490f-bd91-fa8fe7fe8100 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:32:59.998409 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145bf78-a412-44d0-bd7a-4b2a60ebb30e None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:00.034031 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03004c2b-9068-4150-b150-98498bfbb205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:00.034864 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03004c2b-9068-4150-b150-98498bfbb205 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:00.035295 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03004c2b-9068-4150-b150-98498bfbb205 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:00.035565 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03004c2b-9068-4150-b150-98498bfbb205 None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:00.036251 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03004c2b-9068-4150-b150-98498bfbb205 None admin] RBAC: Adding query filter params (name=service) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:00.036551 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03004c2b-9068-4150-b150-98498bfbb205 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:00.037902 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03004c2b-9068-4150-b150-98498bfbb205 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:00.041521 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ce6c37-19ed-490f-bd91-fa8fe7fe8100 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:00.050295 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 211/434] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Oct 17 03:32:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:33:00.050295 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183d8c6-348a-491a-9315-2e12048977df None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:00.050295 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183d8c6-348a-491a-9315-2e12048977df None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:00.050295 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183d8c6-348a-491a-9315-2e12048977df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:00.065959 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-9183d8c6-348a-491a-9315-2e12048977df None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:00.067128 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-9183d8c6-348a-491a-9315-2e12048977df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:00.078538 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c4fd50-8913-4242-bae9-9db764735484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:00.079144 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4fd50-8913-4242-bae9-9db764735484 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:00.079420 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4fd50-8913-4242-bae9-9db764735484 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:00.079688 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4fd50-8913-4242-bae9-9db764735484 None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:00.080342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c4fd50-8913-4242-bae9-9db764735484 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:00.080639 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c4fd50-8913-4242-bae9-9db764735484 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:00.081971 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c4fd50-8913-4242-bae9-9db764735484 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:00.094907 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 224/435] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:32:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:00.096183 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554b68d-ef67-4cf0-83b1-888cac4d5db0 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:00.096466 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554b68d-ef67-4cf0-83b1-888cac4d5db0 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:00.096732 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554b68d-ef67-4cf0-83b1-888cac4d5db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:00.114169 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-f554b68d-ef67-4cf0-83b1-888cac4d5db0 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:00.115357 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-f554b68d-ef67-4cf0-83b1-888cac4d5db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:00.132455 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:00.147970 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 212/436] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:33:00.147970 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183d8c6-348a-491a-9315-2e12048977df None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:00.194398 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 225/437] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:33:00.195465 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554b68d-ef67-4cf0-83b1-888cac4d5db0 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:00.203561 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:00.311243 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183d8c6-348a-491a-9315-2e12048977df None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:00.373122 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554b68d-ef67-4cf0-83b1-888cac4d5db0 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:00.378209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c683add4-8638-4d16-b2aa-19963b1f9e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:00.379025 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683add4-8638-4d16-b2aa-19963b1f9e18 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:00.379904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683add4-8638-4d16-b2aa-19963b1f9e18 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:00.380317 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683add4-8638-4d16-b2aa-19963b1f9e18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:00.398772 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c683add4-8638-4d16-b2aa-19963b1f9e18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:00.400720 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c683add4-8638-4d16-b2aa-19963b1f9e18 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:00.419325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-c683add4-8638-4d16-b2aa-19963b1f9e18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:00.419325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:00.419325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:00.419325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:00.419325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:00.419325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:00.419325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:00.419325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:00.419325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:00.419325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:00.419325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:00.419325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:00.419325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:00.419325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:00.419325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:00.419325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:00.420420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:00.420420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:00.420420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:00.420420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:00.420420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:00.420420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:00.420420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:00.420420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:00.420420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:00.420420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:00.420420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:00.420420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:00.420420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:00.420420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:00.420420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:00.420420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:00.420420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:00.421450 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:00.421450 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:00.421450 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:00.423956 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 213/438] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:00.433259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c683add4-8638-4d16-b2aa-19963b1f9e18 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:00.449239 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f59cf-96d4-4996-b547-31bb6bee7e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:00.449879 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f59cf-96d4-4996-b547-31bb6bee7e12 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:00.450155 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f59cf-96d4-4996-b547-31bb6bee7e12 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:00.450421 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f59cf-96d4-4996-b547-31bb6bee7e12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:00.458372 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f59cf-96d4-4996-b547-31bb6bee7e12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:00.459818 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f59cf-96d4-4996-b547-31bb6bee7e12 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:00.472691 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-1a4f59cf-96d4-4996-b547-31bb6bee7e12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:00.472691 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:00.472691 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:00.472691 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:00.472691 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:00.472691 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:00.472691 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:00.472691 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:00.472691 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:00.472691 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:00.472691 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:00.472691 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:00.472691 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:33:00.472691 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:33:00.472691 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:33:00.475098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:33:00.475098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:00.475098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:00.475098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:00.475098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:00.475098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:00.475098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:00.475098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:00.475098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:00.475098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:00.475098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:00.475098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:00.475098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:00.475098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:00.475098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:00.475098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:00.475098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:00.476035 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:33:00.476035 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:33:00.476035 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:33:00.476035 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:33:00.476035 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:33:00.476035 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:33:00.476035 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:00.476035 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:00.476035 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 226/439] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:33:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:00.476035 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f59cf-96d4-4996-b547-31bb6bee7e12 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:00.496001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:00.548312 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:00.559332 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c683add4-8638-4d16-b2aa-19963b1f9e18 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:00.584906 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f59cf-96d4-4996-b547-31bb6bee7e12 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:00.595651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79eb28-36a8-4bca-af9f-738e84d4591b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:00.596299 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79eb28-36a8-4bca-af9f-738e84d4591b None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:00.596579 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79eb28-36a8-4bca-af9f-738e84d4591b None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:00.596856 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79eb28-36a8-4bca-af9f-738e84d4591b None admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:00.597495 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a79eb28-36a8-4bca-af9f-738e84d4591b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:00.597795 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a79eb28-36a8-4bca-af9f-738e84d4591b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:00.599431 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a79eb28-36a8-4bca-af9f-738e84d4591b None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:00.611534 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 214/440] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:00.612407 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79eb28-36a8-4bca-af9f-738e84d4591b None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:00.623437 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb4d9b-1820-426c-82e4-a67d0f7b22d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:00.624102 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb4d9b-1820-426c-82e4-a67d0f7b22d8 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:00.624384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb4d9b-1820-426c-82e4-a67d0f7b22d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:00.624651 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bb4d9b-1820-426c-82e4-a67d0f7b22d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:00.636545 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bb4d9b-1820-426c-82e4-a67d0f7b22d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:33:00.642572 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-c5bb4d9b-1820-426c-82e4-a67d0f7b22d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:33:00.642572 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:00.642572 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:00.642572 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:00.642572 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:00.642572 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:00.642572 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:00.642572 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:00.642572 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:00.642572 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:00.642572 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:00.642572 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:00.642572 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:33:00.642572 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:33:00.642572 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:33:00.642572 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:33:00.644363 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:33:00.644363 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:33:00.644363 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:33:00.644363 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:33:00.644363 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:00.644363 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:00.644363 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:00.644363 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:00.644363 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:00.644363 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:00.644363 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:00.644363 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:00.644363 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:00.644363 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:00.644363 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:00.644363 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:00.645204 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:00.645204 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:00.645204 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:00.645204 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:00.645204 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:33:00.645204 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:33:00.645204 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:33:00.645204 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:33:00.645204 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:33:00.645204 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:33:00.645204 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:33:00.645204 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:00.645204 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 227/441] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Oct 17 03:33:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:00.645204 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb4d9b-1820-426c-82e4-a67d0f7b22d8 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:00.726841 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79eb28-36a8-4bca-af9f-738e84d4591b None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:00.727071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:00.759650 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bb4d9b-1820-426c-82e4-a67d0f7b22d8 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:00.762699 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c1711-2191-4188-90c3-442a2851e9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:00.763335 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c1711-2191-4188-90c3-442a2851e9a7 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:00.763609 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c1711-2191-4188-90c3-442a2851e9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:00.763911 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c1711-2191-4188-90c3-442a2851e9a7 None admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:00.764616 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c1711-2191-4188-90c3-442a2851e9a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:00.764920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c1711-2191-4188-90c3-442a2851e9a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:00.766494 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c1711-2191-4188-90c3-442a2851e9a7 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:00.785299 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 215/442] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Oct 17 03:33:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:00.785299 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c1711-2191-4188-90c3-442a2851e9a7 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:00.795915 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:00.808380 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f929b0c-f1ec-4d00-b82b-3a89bafaaeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:00.809109 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f929b0c-f1ec-4d00-b82b-3a89bafaaeeb None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:00.809392 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f929b0c-f1ec-4d00-b82b-3a89bafaaeeb None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:00.809690 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f929b0c-f1ec-4d00-b82b-3a89bafaaeeb None admin] PATH_INFO: `/v3/roles/service` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:00.820828 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f929b0c-f1ec-4d00-b82b-3a89bafaaeeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:33:00.826156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-3f929b0c-f1ec-4d00-b82b-3a89bafaaeeb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:33:00.826156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:00.826156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:00.826156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:00.826156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:00.826156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:00.826156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:00.826156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:00.826156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:00.826156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:00.826156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:00.826156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:00.826156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:33:00.826156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:33:00.826156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:33:00.826156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:33:00.827122 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:33:00.827122 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:33:00.827122 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:33:00.827122 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:33:00.827122 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:00.827122 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:00.827122 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:00.827122 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:00.827122 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:00.827122 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:00.827122 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:00.827122 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:00.827122 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:00.827122 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:00.827122 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:00.827122 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:00.827935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:00.827935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:00.827935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:00.827935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:00.827935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:33:00.827935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:33:00.827935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:33:00.827935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:33:00.827935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:33:00.827935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:33:00.827935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:33:00.827935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:00.827935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 228/443] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Oct 17 03:33:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:00.829909 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f929b0c-f1ec-4d00-b82b-3a89bafaaeeb None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:00.891559 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c1711-2191-4188-90c3-442a2851e9a7 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:00.924524 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8dc2b-e84d-497c-bbf2-17e9b07c06b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:00.925149 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8dc2b-e84d-497c-bbf2-17e9b07c06b1 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:00.925422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8dc2b-e84d-497c-bbf2-17e9b07c06b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:00.925700 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8dc2b-e84d-497c-bbf2-17e9b07c06b1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:00.932469 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca8dc2b-e84d-497c-bbf2-17e9b07c06b1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:00.933809 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f929b0c-f1ec-4d00-b82b-3a89bafaaeeb None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:00.933920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca8dc2b-e84d-497c-bbf2-17e9b07c06b1 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:00.946017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-cca8dc2b-e84d-497c-bbf2-17e9b07c06b1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 17 03:33:00.946017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:00.946017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:00.946017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:00.946017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:00.946017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:00.946017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:00.946017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:00.946017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:00.946017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:00.946017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:00.946017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:00.946017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:33:00.946017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:33:00.946017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:33:00.946017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:33:00.946913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:00.946913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:00.946913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:33:00.946913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:00.946913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:33:00.946913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:00.946913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:00.946913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:00.946913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:00.946913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:00.946913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:00.946913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:00.946913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:00.946913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:00.946913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:00.946913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:00.946913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:00.946913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:00.948362 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:00.948362 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:00.948362 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:33:00.948362 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:33:00.948362 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:33:00.948362 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:33:00.948362 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:33:00.948362 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:33:00.948362 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 17 03:33:00.948362 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:00.948362 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 216/444] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Oct 17 03:33:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:00.948906 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8dc2b-e84d-497c-bbf2-17e9b07c06b1 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:00.967855 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:00.972700 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7dca4-bf10-41b9-aef5-9f2d9ced10a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:00.973309 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7dca4-bf10-41b9-aef5-9f2d9ced10a6 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:00.973587 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7dca4-bf10-41b9-aef5-9f2d9ced10a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:00.973877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7dca4-bf10-41b9-aef5-9f2d9ced10a6 None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:00.974520 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d7dca4-bf10-41b9-aef5-9f2d9ced10a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:00.979793 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d7dca4-bf10-41b9-aef5-9f2d9ced10a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:00.979793 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d7dca4-bf10-41b9-aef5-9f2d9ced10a6 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:00.983940 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 229/445] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Oct 17 03:33:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:00.984726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7dca4-bf10-41b9-aef5-9f2d9ced10a6 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:01.017425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:01.047315 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8dc2b-e84d-497c-bbf2-17e9b07c06b1 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:01.080760 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a9538-56b4-4359-8dfd-c59cad826d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:01.080965 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7dca4-bf10-41b9-aef5-9f2d9ced10a6 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:01.081359 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a9538-56b4-4359-8dfd-c59cad826d29 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:01.081639 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a9538-56b4-4359-8dfd-c59cad826d29 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:01.081937 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a9538-56b4-4359-8dfd-c59cad826d29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:01.082659 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a9538-56b4-4359-8dfd-c59cad826d29 None admin] RBAC: Adding query filter params (role.id=8a0215180fe04d5a8f2c24b370382456, scope.project.id=932cdb9ef7644bf5ba57c6d1d6bce7ff, user.id=d961b751763a4e49b5d4ca4c795c5bf6) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:01.082969 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a9538-56b4-4359-8dfd-c59cad826d29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:01.084440 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a9538-56b4-4359-8dfd-c59cad826d29 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:01.091333 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 217/446] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Oct 17 03:33:00 2022] GET /identity/v3/role_assignments?user.id=d961b751763a4e49b5d4ca4c795c5bf6&scope.project.id=932cdb9ef7644bf5ba57c6d1d6bce7ff&role.id=8a0215180fe04d5a8f2c24b370382456 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:01.092624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789214c-988d-4b55-80c2-561b91fafa2a None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:01.092912 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789214c-988d-4b55-80c2-561b91fafa2a None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:01.093178 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f789214c-988d-4b55-80c2-561b91fafa2a None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:01.096209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 218/447] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:01.096209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f789214c-988d-4b55-80c2-561b91fafa2a None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:01.120353 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd59c6-e203-4d99-b1be-4dbf89f00f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:01.120983 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd59c6-e203-4d99-b1be-4dbf89f00f4c None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:01.121263 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd59c6-e203-4d99-b1be-4dbf89f00f4c None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:01.121561 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd59c6-e203-4d99-b1be-4dbf89f00f4c None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:01.122246 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd59c6-e203-4d99-b1be-4dbf89f00f4c None admin] RBAC: Adding query filter params (name=service) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:01.122567 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd59c6-e203-4d99-b1be-4dbf89f00f4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:01.123973 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd59c6-e203-4d99-b1be-4dbf89f00f4c None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:01.137884 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 230/448] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Oct 17 03:33:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:01.137884 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd59c6-e203-4d99-b1be-4dbf89f00f4c None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:01.198500 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:01.224735 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f789214c-988d-4b55-80c2-561b91fafa2a None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:01.246168 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd59c6-e203-4d99-b1be-4dbf89f00f4c None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:01.264626 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bbacf7-589a-4faf-a77b-06f051412c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:01.265246 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbacf7-589a-4faf-a77b-06f051412c54 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:01.265522 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbacf7-589a-4faf-a77b-06f051412c54 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:01.265798 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bbacf7-589a-4faf-a77b-06f051412c54 None admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:01.266584 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bbacf7-589a-4faf-a77b-06f051412c54 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:01.267427 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bbacf7-589a-4faf-a77b-06f051412c54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:01.269107 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bbacf7-589a-4faf-a77b-06f051412c54 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:01.270149 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:01.289830 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 219/449] 10.222.0.17 () {56 vars in 1135 bytes} [Mon Oct 17 03:33:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:01.289830 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142682b-21d1-477f-a9e3-c6efe24fdad6 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:01.289830 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142682b-21d1-477f-a9e3-c6efe24fdad6 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:01.289830 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142682b-21d1-477f-a9e3-c6efe24fdad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:01.292906 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888068a-0305-4530-ab29-61fa65c40259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:01.293553 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888068a-0305-4530-ab29-61fa65c40259 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:01.293834 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888068a-0305-4530-ab29-61fa65c40259 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:01.294316 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888068a-0305-4530-ab29-61fa65c40259 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:01.306796 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3888068a-0305-4530-ab29-61fa65c40259 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:01.308328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3888068a-0305-4530-ab29-61fa65c40259 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:01.310003 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-1142682b-21d1-477f-a9e3-c6efe24fdad6 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:01.311231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-1142682b-21d1-477f-a9e3-c6efe24fdad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:01.326320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-3888068a-0305-4530-ab29-61fa65c40259 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:01.326320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:01.326320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:01.326320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:01.326320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:01.326320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:01.326320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:01.326320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:01.326320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:01.326320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:01.326320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:01.326320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:01.326320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:01.326320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:01.326320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:01.326320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:01.327202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:01.327202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:01.327202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:01.327202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:01.327202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:01.327202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:01.327202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:01.327202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:01.327202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:01.327202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:01.327202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:01.327202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:01.327202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:01.327202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:01.327202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:01.327202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:01.327202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:01.328606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:01.328606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:01.328606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:01.328606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 231/450] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:01.328606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888068a-0305-4530-ab29-61fa65c40259 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:01.386200 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 220/451] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:33:01.387044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142682b-21d1-477f-a9e3-c6efe24fdad6 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:01.444372 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888068a-0305-4530-ab29-61fa65c40259 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:01.455079 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:01.481945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b68b3-e14c-4377-b058-3430fff4f5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:01.482932 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b68b3-e14c-4377-b058-3430fff4f5cb None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:01.483421 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b68b3-e14c-4377-b058-3430fff4f5cb None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:01.483933 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b68b3-e14c-4377-b058-3430fff4f5cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:01.495215 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b68b3-e14c-4377-b058-3430fff4f5cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:01.497205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b68b3-e14c-4377-b058-3430fff4f5cb None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:01.513303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-6e0b68b3-e14c-4377-b058-3430fff4f5cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:01.513303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:01.513303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:01.513303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:01.513303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:01.513303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:01.513303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:01.513303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:01.513303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:01.513303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:01.513303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:01.513303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:01.513303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:01.513303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:01.513303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:01.513303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:01.514180 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:01.514180 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:01.514180 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:01.514180 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:01.514180 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:01.514180 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:01.514180 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:01.514180 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:01.514180 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:01.514180 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:01.514180 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:01.514180 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:01.514180 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:01.514180 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:01.514180 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:01.514180 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:01.514180 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:01.515007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:01.515007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:01.515007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:01.516951 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 232/452] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:01.517939 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b68b3-e14c-4377-b058-3430fff4f5cb None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:01.524182 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142682b-21d1-477f-a9e3-c6efe24fdad6 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:01.553918 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:01.570399 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47bd403-fa12-4993-a4ea-67d013189c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:01.571664 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bd403-fa12-4993-a4ea-67d013189c2f None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:01.572117 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bd403-fa12-4993-a4ea-67d013189c2f None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:01.573071 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bd403-fa12-4993-a4ea-67d013189c2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:01.587868 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47bd403-fa12-4993-a4ea-67d013189c2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:01.590828 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47bd403-fa12-4993-a4ea-67d013189c2f None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:01.611146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-d47bd403-fa12-4993-a4ea-67d013189c2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:01.611146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:01.611146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:01.611146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:01.611146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:01.611146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:01.611146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:01.611146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:01.611146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:01.611146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:01.611146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:01.611146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:01.611146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:01.611146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:01.611146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:01.611146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:01.612283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:01.612283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:01.612283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:01.612283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:01.612283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:01.612283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:01.612283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:01.612283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:01.612283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:01.612283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:01.612283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:01.612283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:01.612283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:01.612283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:01.612283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:01.612283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:01.612283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:01.613315 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:01.613315 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:01.613315 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:01.614980 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 221/453] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:01.616367 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47bd403-fa12-4993-a4ea-67d013189c2f None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:01.627753 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b68b3-e14c-4377-b058-3430fff4f5cb None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:01.669085 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49dd44-a906-468a-8982-208a03c3bb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:01.670019 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49dd44-a906-468a-8982-208a03c3bb9a None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:01.670555 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49dd44-a906-468a-8982-208a03c3bb9a None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:01.670993 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49dd44-a906-468a-8982-208a03c3bb9a None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:01.671809 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49dd44-a906-468a-8982-208a03c3bb9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:01.672258 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49dd44-a906-468a-8982-208a03c3bb9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:01.673934 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49dd44-a906-468a-8982-208a03c3bb9a None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:01.688185 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 233/454] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:01.689087 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49dd44-a906-468a-8982-208a03c3bb9a None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:01.716189 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:01.745311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47bd403-fa12-4993-a4ea-67d013189c2f None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:01.782118 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d11ed9-73de-475d-b093-6a2f4ea35fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:01.782730 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d11ed9-73de-475d-b093-6a2f4ea35fcf None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:01.783011 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d11ed9-73de-475d-b093-6a2f4ea35fcf None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:01.783282 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d11ed9-73de-475d-b093-6a2f4ea35fcf None admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:01.784023 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d11ed9-73de-475d-b093-6a2f4ea35fcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:01.784323 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d11ed9-73de-475d-b093-6a2f4ea35fcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:01.785713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d11ed9-73de-475d-b093-6a2f4ea35fcf None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:01.796398 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49dd44-a906-468a-8982-208a03c3bb9a None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:01.797820 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 222/455] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:01.798453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d11ed9-73de-475d-b093-6a2f4ea35fcf None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:01.833661 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5352f-b23f-48e5-a684-b5a5df00371b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:01.833867 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:01.834732 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5352f-b23f-48e5-a684-b5a5df00371b None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:01.835644 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5352f-b23f-48e5-a684-b5a5df00371b None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:01.835644 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5352f-b23f-48e5-a684-b5a5df00371b None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:01.835960 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5352f-b23f-48e5-a684-b5a5df00371b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:01.836268 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5352f-b23f-48e5-a684-b5a5df00371b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:01.837664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5352f-b23f-48e5-a684-b5a5df00371b None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:01.850652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 234/456] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:01.851664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5352f-b23f-48e5-a684-b5a5df00371b None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:01.908188 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d11ed9-73de-475d-b093-6a2f4ea35fcf None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:01.943064 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb15b5-6ea9-45c4-ada1-640473c9691c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:01.943693 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb15b5-6ea9-45c4-ada1-640473c9691c None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:01.943997 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb15b5-6ea9-45c4-ada1-640473c9691c None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:01.944270 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb15b5-6ea9-45c4-ada1-640473c9691c None admin] PATH_INFO: `/v3/roles/service` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:01.945359 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:01.955091 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb15b5-6ea9-45c4-ada1-640473c9691c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:33:01.960114 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-c8fb15b5-6ea9-45c4-ada1-640473c9691c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:33:01.960114 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:01.960114 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:01.960114 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:01.960114 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:01.960114 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:01.960114 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:01.960114 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:01.960114 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:01.960114 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:01.960114 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:01.960114 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:01.960114 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:33:01.960114 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:33:01.960114 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:33:01.960114 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:33:01.961150 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:33:01.961150 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:33:01.961150 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:33:01.961150 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:33:01.961150 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:01.961150 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:01.961150 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:01.961150 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:01.961150 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:01.961150 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:01.961150 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:01.961150 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:01.961150 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:01.961150 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:01.961150 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:01.961150 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:01.961943 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:01.961943 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:01.961943 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:01.961943 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:01.961943 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:33:01.961943 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:33:01.961943 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:33:01.961943 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:33:01.961943 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:33:01.961943 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:33:01.961943 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:33:01.961943 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:01.961943 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 223/457] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Oct 17 03:33:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:01.962570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb15b5-6ea9-45c4-ada1-640473c9691c None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:01.970172 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5352f-b23f-48e5-a684-b5a5df00371b None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:02.007311 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1898a2-6d4e-4437-ba49-8336d82c6b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:02.008008 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1898a2-6d4e-4437-ba49-8336d82c6b22 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:02.008296 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1898a2-6d4e-4437-ba49-8336d82c6b22 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:02.008572 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1898a2-6d4e-4437-ba49-8336d82c6b22 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:02.017408 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1898a2-6d4e-4437-ba49-8336d82c6b22 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:02.018869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1898a2-6d4e-4437-ba49-8336d82c6b22 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:02.031152 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-3c1898a2-6d4e-4437-ba49-8336d82c6b22 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 03:33:02.031152 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:02.031152 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:02.031152 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:02.031152 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:02.031152 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:02.031152 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:02.031152 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:02.031152 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:02.031152 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:02.031152 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:02.031152 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:02.031152 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:33:02.031152 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:33:02.031152 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:33:02.031152 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:33:02.032108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:02.032108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:02.032108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:33:02.032108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:02.032108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:33:02.032108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:02.032108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:02.032108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:02.032108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:02.032108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:02.032108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:02.032108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:02.032108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:02.032108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:02.032108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:02.032108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:02.032108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:02.032108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:02.033071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:02.033071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:02.033071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:33:02.033071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:33:02.033071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:33:02.033071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:33:02.033071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:33:02.033071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:33:02.033071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 03:33:02.033071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:02.033071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 235/458] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:33:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:02.033071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1898a2-6d4e-4437-ba49-8336d82c6b22 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:02.066270 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:02.073631 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb15b5-6ea9-45c4-ada1-640473c9691c None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:02.107142 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ab51d-42cf-4d8d-81df-d36ee6ceccf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:02.107796 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ab51d-42cf-4d8d-81df-d36ee6ceccf6 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:02.108085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ab51d-42cf-4d8d-81df-d36ee6ceccf6 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:02.108357 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ab51d-42cf-4d8d-81df-d36ee6ceccf6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:02.115282 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ab51d-42cf-4d8d-81df-d36ee6ceccf6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:02.116792 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5ab51d-42cf-4d8d-81df-d36ee6ceccf6 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:02.128920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-4d5ab51d-42cf-4d8d-81df-d36ee6ceccf6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 17 03:33:02.128920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:02.128920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:02.128920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:02.128920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:02.128920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:02.128920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:02.128920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:02.128920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:02.128920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:02.128920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:02.128920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:02.128920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:33:02.128920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:33:02.128920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:33:02.128920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:33:02.129857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:02.129857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:02.129857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:33:02.129857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:02.129857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:33:02.129857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:02.129857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:02.129857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:02.129857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:02.129857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:02.129857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:02.129857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:02.129857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:02.129857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:02.129857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:02.129857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:02.129857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:02.129857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:02.136126 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:02.136126 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:02.136126 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:33:02.136126 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:33:02.136126 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:33:02.136126 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:33:02.136126 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:33:02.136126 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:33:02.136126 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 17 03:33:02.136126 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:02.136126 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 224/459] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:33:01 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:02.136126 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ab51d-42cf-4d8d-81df-d36ee6ceccf6 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:02.148685 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1898a2-6d4e-4437-ba49-8336d82c6b22 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:02.183775 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75306c35-c54e-42a5-82af-d6877998bae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:02.184385 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75306c35-c54e-42a5-82af-d6877998bae9 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:02.184664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75306c35-c54e-42a5-82af-d6877998bae9 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:02.184936 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75306c35-c54e-42a5-82af-d6877998bae9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:02.191416 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75306c35-c54e-42a5-82af-d6877998bae9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:02.192872 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75306c35-c54e-42a5-82af-d6877998bae9 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:02.195483 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:02.205427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-75306c35-c54e-42a5-82af-d6877998bae9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:02.205427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:02.205427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:02.205427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:02.205427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:02.205427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:02.205427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:02.205427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:02.205427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:02.205427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:02.205427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:02.205427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:02.205427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:33:02.205427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:33:02.205427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:33:02.210111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:33:02.210111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:02.210111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:02.210111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:02.210111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:02.210111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:02.210111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:02.210111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:02.210111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:02.210111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:02.210111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:02.210111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:02.210111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:02.210111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:02.210111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:02.210111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:02.210111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:02.210937 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:33:02.210937 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:33:02.210937 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:33:02.210937 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:33:02.210937 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:33:02.210937 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:33:02.210937 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:02.210937 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:02.210937 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 236/460] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:33:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:02.210937 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75306c35-c54e-42a5-82af-d6877998bae9 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:02.250206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ab51d-42cf-4d8d-81df-d36ee6ceccf6 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:02.284756 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ee80f-195a-43a2-81fb-6fe35e211bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:02.285361 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ee80f-195a-43a2-81fb-6fe35e211bef None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:02.285647 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ee80f-195a-43a2-81fb-6fe35e211bef None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:02.285924 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ee80f-195a-43a2-81fb-6fe35e211bef None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:02.286585 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ee80f-195a-43a2-81fb-6fe35e211bef None admin] RBAC: Adding query filter params (name=service) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:02.286891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ee80f-195a-43a2-81fb-6fe35e211bef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:02.288389 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ee80f-195a-43a2-81fb-6fe35e211bef None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:02.296890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 225/461] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Oct 17 03:33:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:02.297689 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ee80f-195a-43a2-81fb-6fe35e211bef None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:02.317954 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:02.324587 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75306c35-c54e-42a5-82af-d6877998bae9 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:02.360929 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85073fc-e3dd-49c6-a27e-a6a98f1b8bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:02.361546 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85073fc-e3dd-49c6-a27e-a6a98f1b8bc6 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:02.361832 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85073fc-e3dd-49c6-a27e-a6a98f1b8bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:02.362105 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85073fc-e3dd-49c6-a27e-a6a98f1b8bc6 None admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:02.362830 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85073fc-e3dd-49c6-a27e-a6a98f1b8bc6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:02.363138 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85073fc-e3dd-49c6-a27e-a6a98f1b8bc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:02.364582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85073fc-e3dd-49c6-a27e-a6a98f1b8bc6 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:02.383317 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 237/462] 10.222.0.17 () {56 vars in 1137 bytes} [Mon Oct 17 03:33:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:02.384169 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85073fc-e3dd-49c6-a27e-a6a98f1b8bc6 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:02.412475 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ee80f-195a-43a2-81fb-6fe35e211bef None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:02.445698 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:02.448753 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c705f3e-a01e-4d7b-8f68-f6ec327753ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:02.449367 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c705f3e-a01e-4d7b-8f68-f6ec327753ad None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:02.449642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c705f3e-a01e-4d7b-8f68-f6ec327753ad None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:02.450085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c705f3e-a01e-4d7b-8f68-f6ec327753ad None admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:02.450646 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c705f3e-a01e-4d7b-8f68-f6ec327753ad None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:02.450986 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c705f3e-a01e-4d7b-8f68-f6ec327753ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:02.452438 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c705f3e-a01e-4d7b-8f68-f6ec327753ad None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:02.470662 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 226/463] 10.222.0.17 () {56 vars in 1137 bytes} [Mon Oct 17 03:33:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:02.471914 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c705f3e-a01e-4d7b-8f68-f6ec327753ad None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:02.489889 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85073fc-e3dd-49c6-a27e-a6a98f1b8bc6 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:02.525143 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1109727-c134-435d-8196-56e263f1901c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:02.525699 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1109727-c134-435d-8196-56e263f1901c None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:02.525973 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1109727-c134-435d-8196-56e263f1901c None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:02.526238 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1109727-c134-435d-8196-56e263f1901c None admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:02.526916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1109727-c134-435d-8196-56e263f1901c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:02.527211 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1109727-c134-435d-8196-56e263f1901c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:02.528641 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1109727-c134-435d-8196-56e263f1901c None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:02.539675 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 238/464] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Oct 17 03:33:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:02.544870 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1109727-c134-435d-8196-56e263f1901c None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:02.561009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:02.586451 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c705f3e-a01e-4d7b-8f68-f6ec327753ad None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:02.623180 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b914655-2026-41b5-bdaf-1a4362c7c640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:02.623829 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b914655-2026-41b5-bdaf-1a4362c7c640 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:02.624111 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b914655-2026-41b5-bdaf-1a4362c7c640 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:02.624382 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b914655-2026-41b5-bdaf-1a4362c7c640 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:02.634883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b914655-2026-41b5-bdaf-1a4362c7c640 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:02.636768 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b914655-2026-41b5-bdaf-1a4362c7c640 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:02.648592 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1109727-c134-435d-8196-56e263f1901c None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:02.654870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-6b914655-2026-41b5-bdaf-1a4362c7c640 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:02.654870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:02.654870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:02.654870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:02.654870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:02.654870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:02.654870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:02.654870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:02.654870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:02.654870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:02.654870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:02.654870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:02.654870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:02.654870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:02.654870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:02.654870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:02.655990 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:02.655990 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:02.655990 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:02.655990 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:02.655990 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:02.655990 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:02.655990 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:02.655990 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:02.655990 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:02.655990 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:02.655990 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:02.655990 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:02.655990 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:02.655990 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:02.655990 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:02.655990 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:02.655990 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:02.656864 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:02.656864 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:02.656864 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:02.656864 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 227/465] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:02.657074 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b914655-2026-41b5-bdaf-1a4362c7c640 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:02.668989 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:02.690042 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44815311-bb77-47a1-8fc3-0c131591644c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:02.690665 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44815311-bb77-47a1-8fc3-0c131591644c None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:02.691271 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44815311-bb77-47a1-8fc3-0c131591644c None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:02.691549 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44815311-bb77-47a1-8fc3-0c131591644c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:02.766938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44815311-bb77-47a1-8fc3-0c131591644c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:02.768417 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44815311-bb77-47a1-8fc3-0c131591644c None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:02.768636 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b914655-2026-41b5-bdaf-1a4362c7c640 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:02.795352 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:02.803441 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5551e1-e559-4970-bfe3-049e3d6d5412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:02.808682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5551e1-e559-4970-bfe3-049e3d6d5412 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:02.808682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5551e1-e559-4970-bfe3-049e3d6d5412 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:02.808682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5551e1-e559-4970-bfe3-049e3d6d5412 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:02.875370 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5551e1-e559-4970-bfe3-049e3d6d5412 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:02.876867 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5551e1-e559-4970-bfe3-049e3d6d5412 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:02.877340 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 239/466] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:33:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 03:33:02.878334 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44815311-bb77-47a1-8fc3-0c131591644c None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:02.899668 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:02.985729 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 228/467] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:33:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 03:33:02.986584 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5551e1-e559-4970-bfe3-049e3d6d5412 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:02.995509 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44815311-bb77-47a1-8fc3-0c131591644c None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:03.014727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:03.031659 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3896bf3-4f33-4b86-ab2a-fac0b0fa0430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:03.032602 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3896bf3-4f33-4b86-ab2a-fac0b0fa0430 None admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:03.032808 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3896bf3-4f33-4b86-ab2a-fac0b0fa0430 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:03.033080 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3896bf3-4f33-4b86-ab2a-fac0b0fa0430 None admin] PATH_INFO: `/v3/projects/932cdb9ef7644bf5ba57c6d1d6bce7ff/users/5f64aaf4d408462a88963973d42abfb9/roles/8a0215180fe04d5a8f2c24b370382456` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:03.048296 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3896bf3-4f33-4b86-ab2a-fac0b0fa0430 None admin] RBAC: Authorizing `identity:create_grant(project_id=932cdb9ef7644bf5ba57c6d1d6bce7ff, user_id=5f64aaf4d408462a88963973d42abfb9, role_id=8a0215180fe04d5a8f2c24b370382456)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:03.049653 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3896bf3-4f33-4b86-ab2a-fac0b0fa0430 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:03.087667 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5551e1-e559-4970-bfe3-049e3d6d5412 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:03.122766 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed25d08-779e-41a0-ace4-16241c453806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:03.123374 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed25d08-779e-41a0-ace4-16241c453806 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:03.123654 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed25d08-779e-41a0-ace4-16241c453806 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:03.123975 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed25d08-779e-41a0-ace4-16241c453806 None admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:03.124616 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed25d08-779e-41a0-ace4-16241c453806 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:03.124978 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed25d08-779e-41a0-ace4-16241c453806 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:03.126302 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed25d08-779e-41a0-ace4-16241c453806 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:03.139766 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 240/468] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Oct 17 03:33:02 2022] PUT /identity/v3/projects/932cdb9ef7644bf5ba57c6d1d6bce7ff/users/5f64aaf4d408462a88963973d42abfb9/roles/8a0215180fe04d5a8f2c24b370382456 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:33:03.139766 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3896bf3-4f33-4b86-ab2a-fac0b0fa0430 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:03.149355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 229/469] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:03.152067 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed25d08-779e-41a0-ace4-16241c453806 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:03.228599 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:03.266181 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:03.267940 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3896bf3-4f33-4b86-ab2a-fac0b0fa0430 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:03.288006 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed25d08-779e-41a0-ace4-16241c453806 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:03.308731 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd962510-b68a-4edb-9ea9-fcc85e0f67b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:03.309631 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd962510-b68a-4edb-9ea9-fcc85e0f67b1 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:03.310059 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd962510-b68a-4edb-9ea9-fcc85e0f67b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:03.310497 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd962510-b68a-4edb-9ea9-fcc85e0f67b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:03.324206 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd962510-b68a-4edb-9ea9-fcc85e0f67b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:03.326322 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd962510-b68a-4edb-9ea9-fcc85e0f67b1 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:03.326518 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ccc38-f236-4492-b36c-2d41602ea327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:03.327169 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ccc38-f236-4492-b36c-2d41602ea327 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:03.327449 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ccc38-f236-4492-b36c-2d41602ea327 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:03.327761 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ccc38-f236-4492-b36c-2d41602ea327 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:03.340807 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ccc38-f236-4492-b36c-2d41602ea327 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:03.342252 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ccc38-f236-4492-b36c-2d41602ea327 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:03.344837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-cd962510-b68a-4edb-9ea9-fcc85e0f67b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:03.344837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:03.344837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:03.344837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:03.344837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:03.344837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:03.344837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:03.344837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:03.344837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:03.344837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:03.344837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:03.344837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:03.344837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:03.344837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:03.344837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:03.344837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:03.345696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:03.345696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:03.345696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:03.345696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:03.345696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:03.345696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:03.345696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:03.345696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:03.345696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:03.345696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:03.345696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:03.345696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:03.345696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:03.345696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:03.345696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:03.345696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:03.345696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:03.346527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:03.346527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:03.346527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:03.346527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 241/470] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:03.346735 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd962510-b68a-4edb-9ea9-fcc85e0f67b1 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:03.359106 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-3a0ccc38-f236-4492-b36c-2d41602ea327 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:03.359106 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:03.359106 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:03.359106 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:03.359106 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:03.359106 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:03.359106 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:03.359106 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:03.359106 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:03.359106 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:03.359106 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:03.359106 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:03.359106 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:03.359106 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:03.359106 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:03.359106 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:03.360058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:03.360058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:03.360058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:03.360058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:03.360058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:03.360058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:03.360058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:03.360058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:03.360058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:03.360058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:03.360058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:03.360058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:03.360058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:03.360058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:03.360058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:03.360058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:03.360058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:03.361006 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:03.361006 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:03.361006 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:03.361006 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 230/471] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:03.362667 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d019b4a-4118-4bec-90fb-f5ae41da23b1 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:03.362946 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d019b4a-4118-4bec-90fb-f5ae41da23b1 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:03.363217 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d019b4a-4118-4bec-90fb-f5ae41da23b1 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:03.367836 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 231/472] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:03.369710 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d019b4a-4118-4bec-90fb-f5ae41da23b1 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:03.449150 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd962510-b68a-4edb-9ea9-fcc85e0f67b1 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:03.470535 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:03.476977 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d019b4a-4118-4bec-90fb-f5ae41da23b1 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:03.486074 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13eae9-0189-4bbb-b1c2-8ba8b455f4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:03.487061 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13eae9-0189-4bbb-b1c2-8ba8b455f4e0 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:03.487343 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13eae9-0189-4bbb-b1c2-8ba8b455f4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:03.487616 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13eae9-0189-4bbb-b1c2-8ba8b455f4e0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:03.496107 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13eae9-0189-4bbb-b1c2-8ba8b455f4e0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:03.497554 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13eae9-0189-4bbb-b1c2-8ba8b455f4e0 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:03.510430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-6b13eae9-0189-4bbb-b1c2-8ba8b455f4e0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 17 03:33:03.510430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:03.510430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:03.510430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:03.510430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:03.510430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:03.510430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:03.510430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:03.510430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:03.510430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:03.510430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:03.510430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:03.510430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:33:03.510430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:33:03.510430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:33:03.510430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:33:03.511302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:03.511302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:03.511302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:33:03.511302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:03.511302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:33:03.511302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:03.511302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:03.511302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:03.511302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:03.511302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:03.511302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:03.511302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:03.511302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:03.511302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:03.511302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:03.511302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:03.511302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:03.511302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:03.512218 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:03.512218 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:03.512218 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:33:03.512218 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:33:03.512218 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:33:03.512218 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:33:03.512218 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:33:03.512218 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:33:03.512218 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 17 03:33:03.512218 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:03.512218 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 242/473] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Oct 17 03:33:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:03.512758 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13eae9-0189-4bbb-b1c2-8ba8b455f4e0 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:03.519045 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4de590-3efa-42a7-ac44-4bdf6194fcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:03.519662 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4de590-3efa-42a7-ac44-4bdf6194fcd9 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:03.519971 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4de590-3efa-42a7-ac44-4bdf6194fcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:03.520242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4de590-3efa-42a7-ac44-4bdf6194fcd9 None admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:03.520883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4de590-3efa-42a7-ac44-4bdf6194fcd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:03.521184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4de590-3efa-42a7-ac44-4bdf6194fcd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:03.522536 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4de590-3efa-42a7-ac44-4bdf6194fcd9 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:03.530131 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:03.539778 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 232/474] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:03.539778 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8183c-d66d-411a-81aa-4bb5ba3aee49 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:03.539778 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8183c-d66d-411a-81aa-4bb5ba3aee49 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:03.539778 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8183c-d66d-411a-81aa-4bb5ba3aee49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:03.555148 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-bfc8183c-d66d-411a-81aa-4bb5ba3aee49 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:03.556344 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-bfc8183c-d66d-411a-81aa-4bb5ba3aee49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:03.623286 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13eae9-0189-4bbb-b1c2-8ba8b455f4e0 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:03.626093 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 233/475] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:33:03.626933 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8183c-d66d-411a-81aa-4bb5ba3aee49 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:03.672738 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4ee37-47eb-48f8-b45f-7e7d012644dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:03.673678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4ee37-47eb-48f8-b45f-7e7d012644dc None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:03.673971 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4ee37-47eb-48f8-b45f-7e7d012644dc None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:03.674541 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c4ee37-47eb-48f8-b45f-7e7d012644dc None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:03.675570 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4ee37-47eb-48f8-b45f-7e7d012644dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:03.676032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4ee37-47eb-48f8-b45f-7e7d012644dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:03.679054 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c4ee37-47eb-48f8-b45f-7e7d012644dc None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:03.695913 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 243/476] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:03.697668 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4ee37-47eb-48f8-b45f-7e7d012644dc None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:03.755614 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:03.768621 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8183c-d66d-411a-81aa-4bb5ba3aee49 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:03.805591 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:03.814408 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd65108-faad-4374-8169-73966c7d78f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:03.815035 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd65108-faad-4374-8169-73966c7d78f5 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:03.815312 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd65108-faad-4374-8169-73966c7d78f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:03.815589 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd65108-faad-4374-8169-73966c7d78f5 None admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:03.816358 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd65108-faad-4374-8169-73966c7d78f5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:03.816671 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd65108-faad-4374-8169-73966c7d78f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:03.818152 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd65108-faad-4374-8169-73966c7d78f5 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:03.829203 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c4ee37-47eb-48f8-b45f-7e7d012644dc None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:03.841896 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 234/477] 10.222.0.17 () {56 vars in 1139 bytes} [Mon Oct 17 03:33:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:03.841896 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd65108-faad-4374-8169-73966c7d78f5 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:03.879513 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13d3d9-62bf-4131-9abb-204750e68960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:03.880263 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13d3d9-62bf-4131-9abb-204750e68960 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:03.881106 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13d3d9-62bf-4131-9abb-204750e68960 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:03.881483 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13d3d9-62bf-4131-9abb-204750e68960 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:03.891275 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13d3d9-62bf-4131-9abb-204750e68960 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:03.893022 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13d3d9-62bf-4131-9abb-204750e68960 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:03.908839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-aa13d3d9-62bf-4131-9abb-204750e68960 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:03.908839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:03.908839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:03.908839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:03.908839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:03.908839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:03.908839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:03.908839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:03.908839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:03.908839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:03.908839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:03.908839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:03.908839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:33:03.908839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:33:03.908839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:33:03.909899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:33:03.909899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:03.909899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:03.909899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:03.909899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:03.909899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:03.909899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:03.909899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:03.909899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:03.909899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:03.909899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:03.909899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:03.909899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:03.909899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:03.909899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:03.909899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:03.909899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:03.910985 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:33:03.910985 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:33:03.910985 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:33:03.910985 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:33:03.910985 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:33:03.910985 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:33:03.910985 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:03.910985 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:03.911890 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 244/478] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:33:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:03.912906 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13d3d9-62bf-4131-9abb-204750e68960 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:03.966137 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd65108-faad-4374-8169-73966c7d78f5 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:04.000731 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a275ad-8144-4cb7-89b3-e9d6f2712793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:04.001586 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a275ad-8144-4cb7-89b3-e9d6f2712793 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:04.001860 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a275ad-8144-4cb7-89b3-e9d6f2712793 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:04.002138 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a275ad-8144-4cb7-89b3-e9d6f2712793 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:04.008622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a275ad-8144-4cb7-89b3-e9d6f2712793 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:04.010079 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a275ad-8144-4cb7-89b3-e9d6f2712793 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:04.021161 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-04a275ad-8144-4cb7-89b3-e9d6f2712793 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:04.021161 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:04.021161 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:04.021161 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:04.021161 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:04.021161 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:04.021161 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:04.021161 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:04.021161 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:04.021161 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:04.021161 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:04.021161 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:04.021161 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:33:04.021161 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:33:04.021161 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:33:04.021985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:33:04.021985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:04.021985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:04.021985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:04.021985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:04.021985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:04.021985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:04.021985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:04.021985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:04.021985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:04.021985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:04.021985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:04.021985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:04.021985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:04.021985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:04.021985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:04.021985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:04.022798 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:33:04.022798 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:33:04.022798 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:33:04.022798 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:33:04.022798 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:33:04.022798 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:33:04.022798 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:04.022798 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:04.022798 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 235/479] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:33:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:04.023245 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a275ad-8144-4cb7-89b3-e9d6f2712793 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:04.024475 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:04.029412 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13d3d9-62bf-4131-9abb-204750e68960 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:04.066582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c368974-713a-40e2-9106-9f10ead46b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:04.067376 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c368974-713a-40e2-9106-9f10ead46b6f None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:04.067780 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c368974-713a-40e2-9106-9f10ead46b6f None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:04.068422 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c368974-713a-40e2-9106-9f10ead46b6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:04.073381 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:04.129776 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a275ad-8144-4cb7-89b3-e9d6f2712793 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:04.143183 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c368974-713a-40e2-9106-9f10ead46b6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:04.144729 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c368974-713a-40e2-9106-9f10ead46b6f None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:04.172094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7266a07-accb-4a16-bf2c-473077c6e053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:04.173190 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7266a07-accb-4a16-bf2c-473077c6e053 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:04.173612 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7266a07-accb-4a16-bf2c-473077c6e053 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:04.174164 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7266a07-accb-4a16-bf2c-473077c6e053 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:04.186635 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7266a07-accb-4a16-bf2c-473077c6e053 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:33:04.193637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-a7266a07-accb-4a16-bf2c-473077c6e053 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:33:04.193637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:04.193637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:04.193637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:04.193637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:04.193637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:04.193637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:04.193637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:04.193637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:04.193637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:04.193637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:04.193637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:04.193637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:33:04.193637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:33:04.193637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:33:04.193637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:33:04.194660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:33:04.194660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:33:04.194660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:33:04.194660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:33:04.194660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:04.194660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:04.194660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:04.194660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:04.194660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:04.194660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:04.194660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:04.194660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:04.194660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:04.194660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:04.194660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:04.194660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:04.195629 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:04.195629 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:04.195629 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:04.195629 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:04.195629 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:33:04.195629 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:33:04.195629 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:33:04.195629 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:33:04.195629 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:33:04.195629 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:33:04.195629 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:33:04.195629 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:04.202391 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 236/480] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Oct 17 03:33:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:04.202391 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7266a07-accb-4a16-bf2c-473077c6e053 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:04.249627 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:04.253285 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 245/481] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:33:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 03:33:04.253977 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c368974-713a-40e2-9106-9f10ead46b6f None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:04.307542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7266a07-accb-4a16-bf2c-473077c6e053 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:04.314570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:04.342468 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c57c91a-47b2-4c25-92a6-f17e979c9003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:04.343567 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57c91a-47b2-4c25-92a6-f17e979c9003 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:04.343874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57c91a-47b2-4c25-92a6-f17e979c9003 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:04.344147 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57c91a-47b2-4c25-92a6-f17e979c9003 None admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:04.344830 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c57c91a-47b2-4c25-92a6-f17e979c9003 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:04.345140 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c57c91a-47b2-4c25-92a6-f17e979c9003 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:04.346550 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c57c91a-47b2-4c25-92a6-f17e979c9003 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:04.358739 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 237/482] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Oct 17 03:33:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:04.359050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c368974-713a-40e2-9106-9f10ead46b6f None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:04.359596 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c57c91a-47b2-4c25-92a6-f17e979c9003 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:04.402495 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e73696b-ca0d-47d1-9739-72ac295a4a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:04.403781 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73696b-ca0d-47d1-9739-72ac295a4a53 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:04.404405 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73696b-ca0d-47d1-9739-72ac295a4a53 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:04.405198 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e73696b-ca0d-47d1-9739-72ac295a4a53 None admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:04.406396 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e73696b-ca0d-47d1-9739-72ac295a4a53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:04.406885 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e73696b-ca0d-47d1-9739-72ac295a4a53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:04.408673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e73696b-ca0d-47d1-9739-72ac295a4a53 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:04.422947 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 246/483] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Oct 17 03:33:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:04.424099 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e73696b-ca0d-47d1-9739-72ac295a4a53 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:04.469211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c57c91a-47b2-4c25-92a6-f17e979c9003 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:04.490568 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:04.502536 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5b1da-8461-4245-ab64-92a1a9acb665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:04.503140 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5b1da-8461-4245-ab64-92a1a9acb665 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:04.503415 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5b1da-8461-4245-ab64-92a1a9acb665 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:04.503688 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5b1da-8461-4245-ab64-92a1a9acb665 None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:04.504369 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5b1da-8461-4245-ab64-92a1a9acb665 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:04.504671 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5b1da-8461-4245-ab64-92a1a9acb665 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:04.506029 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5b1da-8461-4245-ab64-92a1a9acb665 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:04.514467 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 238/484] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:33:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:04.515239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5b1da-8461-4245-ab64-92a1a9acb665 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:04.541734 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e73696b-ca0d-47d1-9739-72ac295a4a53 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:04.544728 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:04.581635 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d247decd-0f69-463a-ba64-85e3ff017dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:04.582269 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247decd-0f69-463a-ba64-85e3ff017dc2 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:04.582539 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247decd-0f69-463a-ba64-85e3ff017dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:04.582963 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d247decd-0f69-463a-ba64-85e3ff017dc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:04.594934 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d247decd-0f69-463a-ba64-85e3ff017dc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:04.596393 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d247decd-0f69-463a-ba64-85e3ff017dc2 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:04.610547 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-d247decd-0f69-463a-ba64-85e3ff017dc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:04.610547 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:04.610547 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:04.610547 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:04.610547 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:04.610547 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:04.610547 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:04.610547 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:04.610547 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:04.610547 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:04.610547 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:04.610547 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:04.610547 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:04.610547 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:04.610547 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:04.610547 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:04.611432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:04.611432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:04.611432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:04.611432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:04.611432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:04.611432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:04.611432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:04.611432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:04.611432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:04.611432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:04.611432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:04.611432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:04.611432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:04.611432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:04.611432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:04.611432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:04.611432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:04.612306 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:04.612306 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:04.612306 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:04.612306 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 247/485] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:04.612520 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d247decd-0f69-463a-ba64-85e3ff017dc2 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:04.622702 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5b1da-8461-4245-ab64-92a1a9acb665 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:04.659331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5aa663-3d90-4fae-9172-5404006005c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:04.660162 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5aa663-3d90-4fae-9172-5404006005c1 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:04.660447 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5aa663-3d90-4fae-9172-5404006005c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:04.660745 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5aa663-3d90-4fae-9172-5404006005c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:04.661471 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5aa663-3d90-4fae-9172-5404006005c1 None admin] RBAC: Adding query filter params (role.id=8a0215180fe04d5a8f2c24b370382456, scope.project.id=932cdb9ef7644bf5ba57c6d1d6bce7ff, user.id=99bc94105bbf4cb4a60e37133edc8dfb) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:04.661781 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5aa663-3d90-4fae-9172-5404006005c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:04.663223 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5aa663-3d90-4fae-9172-5404006005c1 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:04.668956 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 239/486] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Oct 17 03:33:04 2022] GET /identity/v3/role_assignments?user.id=99bc94105bbf4cb4a60e37133edc8dfb&scope.project.id=932cdb9ef7644bf5ba57c6d1d6bce7ff&role.id=8a0215180fe04d5a8f2c24b370382456 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:04.669738 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5aa663-3d90-4fae-9172-5404006005c1 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:04.747362 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d247decd-0f69-463a-ba64-85e3ff017dc2 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:04.767903 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:04.776793 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:04.791691 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5aa663-3d90-4fae-9172-5404006005c1 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:04.793596 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3cdf63-bcdb-46d6-90ff-9109e3cfb2d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:04.794204 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3cdf63-bcdb-46d6-90ff-9109e3cfb2d9 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:04.794470 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3cdf63-bcdb-46d6-90ff-9109e3cfb2d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:04.794739 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3cdf63-bcdb-46d6-90ff-9109e3cfb2d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:04.795770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3cdf63-bcdb-46d6-90ff-9109e3cfb2d9 None admin] RBAC: Adding query filter params (role.id=8a0215180fe04d5a8f2c24b370382456, scope.project.id=932cdb9ef7644bf5ba57c6d1d6bce7ff, user.id=565a836c7f89478cad615bd6c14531a7) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:04.795770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3cdf63-bcdb-46d6-90ff-9109e3cfb2d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:04.797225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3cdf63-bcdb-46d6-90ff-9109e3cfb2d9 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:04.808676 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 248/487] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Oct 17 03:33:04 2022] GET /identity/v3/role_assignments?user.id=565a836c7f89478cad615bd6c14531a7&scope.project.id=932cdb9ef7644bf5ba57c6d1d6bce7ff&role.id=8a0215180fe04d5a8f2c24b370382456 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:04.809838 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3cdf63-bcdb-46d6-90ff-9109e3cfb2d9 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:04.844991 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825bdad-430e-4ceb-a596-f0da724d06ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:04.846597 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825bdad-430e-4ceb-a596-f0da724d06ca None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:04.847049 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825bdad-430e-4ceb-a596-f0da724d06ca None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:04.847471 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1825bdad-430e-4ceb-a596-f0da724d06ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:04.874786 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1825bdad-430e-4ceb-a596-f0da724d06ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:04.878080 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1825bdad-430e-4ceb-a596-f0da724d06ca None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:04.896308 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-1825bdad-430e-4ceb-a596-f0da724d06ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:04.896308 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:04.896308 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:04.896308 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:04.896308 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:04.896308 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:04.896308 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:04.896308 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:04.896308 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:04.896308 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:04.896308 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:04.896308 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:04.896308 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:04.896308 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:04.896308 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:04.896308 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:04.897248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:04.897248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:04.897248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:04.897248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:04.897248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:04.897248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:04.897248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:04.897248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:04.897248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:04.897248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:04.897248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:04.897248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:04.897248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:04.897248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:04.897248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:04.897248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:04.897248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:04.898068 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:04.898068 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:04.898068 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:04.900441 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 240/488] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:04.906540 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825bdad-430e-4ceb-a596-f0da724d06ca None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:04.994796 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3cdf63-bcdb-46d6-90ff-9109e3cfb2d9 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:05.044980 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427095b9-c605-4c3d-807f-ed52983636df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:05.045869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427095b9-c605-4c3d-807f-ed52983636df None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:05.046158 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427095b9-c605-4c3d-807f-ed52983636df None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:05.046435 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427095b9-c605-4c3d-807f-ed52983636df None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:05.047088 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427095b9-c605-4c3d-807f-ed52983636df None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:05.047395 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427095b9-c605-4c3d-807f-ed52983636df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:05.048645 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1825bdad-430e-4ceb-a596-f0da724d06ca None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:05.049589 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427095b9-c605-4c3d-807f-ed52983636df None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:05.067528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 249/489] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:05.071424 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427095b9-c605-4c3d-807f-ed52983636df None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:05.099297 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4597d5-0c35-4e03-99cc-aa18ab801219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:05.100683 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4597d5-0c35-4e03-99cc-aa18ab801219 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:05.101115 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4597d5-0c35-4e03-99cc-aa18ab801219 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:05.101782 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4597d5-0c35-4e03-99cc-aa18ab801219 None admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:05.102803 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4597d5-0c35-4e03-99cc-aa18ab801219 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:05.103773 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4597d5-0c35-4e03-99cc-aa18ab801219 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:05.105607 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4597d5-0c35-4e03-99cc-aa18ab801219 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:05.117873 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:05.122977 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 241/490] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:05.128847 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4597d5-0c35-4e03-99cc-aa18ab801219 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:05.140075 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:05.195634 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427095b9-c605-4c3d-807f-ed52983636df None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:05.236553 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91590612-d0ae-4013-ae68-c0d1a8b62582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:05.237288 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91590612-d0ae-4013-ae68-c0d1a8b62582 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:05.237659 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91590612-d0ae-4013-ae68-c0d1a8b62582 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:05.238155 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91590612-d0ae-4013-ae68-c0d1a8b62582 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:05.246335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91590612-d0ae-4013-ae68-c0d1a8b62582 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:05.247835 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91590612-d0ae-4013-ae68-c0d1a8b62582 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:05.253275 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4597d5-0c35-4e03-99cc-aa18ab801219 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:05.262033 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-91590612-d0ae-4013-ae68-c0d1a8b62582 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:05.262033 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:05.262033 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:05.262033 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:05.262033 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:05.262033 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:05.262033 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:05.262033 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:05.262033 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:05.262033 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:05.262033 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:05.262033 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:05.262033 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:33:05.262033 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:33:05.262033 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:33:05.263119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:33:05.263119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:05.263119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:05.263119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:05.263119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:05.263119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:05.263119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:05.263119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:05.263119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:05.263119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:05.263119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:05.263119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:05.263119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:05.263119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:05.263119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:05.263119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:05.263119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:05.264890 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:33:05.264890 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:33:05.264890 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:33:05.264890 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:33:05.264890 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:33:05.264890 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:33:05.264890 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:05.264890 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:05.264890 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 250/491] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:33:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:05.276966 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91590612-d0ae-4013-ae68-c0d1a8b62582 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:05.299172 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a1b36-f0f5-4d64-bc28-4e742f36b0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:05.299865 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a1b36-f0f5-4d64-bc28-4e742f36b0ce None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:05.300146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a1b36-f0f5-4d64-bc28-4e742f36b0ce None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:05.300601 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488a1b36-f0f5-4d64-bc28-4e742f36b0ce None admin] PATH_INFO: `/v3/users/nova` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:05.308646 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a1b36-f0f5-4d64-bc28-4e742f36b0ce None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:05.310457 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488a1b36-f0f5-4d64-bc28-4e742f36b0ce None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:05.324515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-488a1b36-f0f5-4d64-bc28-4e742f36b0ce None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 03:33:05.324515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:05.324515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:05.324515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:05.324515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:05.324515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:05.324515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:05.324515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:05.324515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:05.324515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:05.324515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:05.324515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:05.324515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:33:05.324515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:33:05.324515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:33:05.324515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:33:05.325648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:05.325648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:05.325648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:33:05.325648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:05.325648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:33:05.325648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:05.325648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:05.325648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:05.325648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:05.325648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:05.325648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:05.325648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:05.325648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:05.325648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:05.325648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:05.325648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:05.325648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:05.325648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:05.326890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:05.326890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:05.326890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:33:05.326890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:33:05.326890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:33:05.326890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:33:05.326890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:33:05.326890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:33:05.326890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 03:33:05.326890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:05.326890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 242/492] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Oct 17 03:33:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:05.331239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a1b36-f0f5-4d64-bc28-4e742f36b0ce None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:05.401736 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:05.442392 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488a1b36-f0f5-4d64-bc28-4e742f36b0ce None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:05.465737 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:05.472915 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91590612-d0ae-4013-ae68-c0d1a8b62582 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:05.480153 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dffd81b-18a7-44b9-be4a-1e094aa1565f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:05.480764 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffd81b-18a7-44b9-be4a-1e094aa1565f None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:05.481045 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffd81b-18a7-44b9-be4a-1e094aa1565f None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:05.481370 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dffd81b-18a7-44b9-be4a-1e094aa1565f None admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:05.482105 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dffd81b-18a7-44b9-be4a-1e094aa1565f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:05.482462 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dffd81b-18a7-44b9-be4a-1e094aa1565f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:05.483928 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dffd81b-18a7-44b9-be4a-1e094aa1565f None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:05.505471 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 243/493] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Oct 17 03:33:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:33:05.506775 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49624e-cb7c-4067-b300-1189f7070f48 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:05.507093 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49624e-cb7c-4067-b300-1189f7070f48 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:05.507365 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49624e-cb7c-4067-b300-1189f7070f48 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:05.513387 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 244/494] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:05.514300 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49624e-cb7c-4067-b300-1189f7070f48 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:05.516638 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d50d56-20d2-4a9e-9268-321f143d7af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:05.517427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d50d56-20d2-4a9e-9268-321f143d7af4 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:05.517705 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d50d56-20d2-4a9e-9268-321f143d7af4 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:05.518784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d50d56-20d2-4a9e-9268-321f143d7af4 None admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:05.519491 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d50d56-20d2-4a9e-9268-321f143d7af4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:05.519841 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d50d56-20d2-4a9e-9268-321f143d7af4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:05.521273 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d50d56-20d2-4a9e-9268-321f143d7af4 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:05.537107 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 251/495] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Oct 17 03:33:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:05.539129 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd65536-99fa-4b57-b553-d76002bf7beb None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:05.539418 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd65536-99fa-4b57-b553-d76002bf7beb None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:05.539689 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd65536-99fa-4b57-b553-d76002bf7beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:05.561889 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-4fd65536-99fa-4b57-b553-d76002bf7beb None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:05.563501 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-4fd65536-99fa-4b57-b553-d76002bf7beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:05.628288 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49624e-cb7c-4067-b300-1189f7070f48 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:05.654024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 252/496] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:33:05.654525 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:05.655903 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd65536-99fa-4b57-b553-d76002bf7beb None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:05.680431 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c21d68-5c52-44bf-ac84-2c78a7151453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:05.681263 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c21d68-5c52-44bf-ac84-2c78a7151453 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:05.682154 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c21d68-5c52-44bf-ac84-2c78a7151453 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:05.682492 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c21d68-5c52-44bf-ac84-2c78a7151453 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:05.706656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c21d68-5c52-44bf-ac84-2c78a7151453 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:05.708478 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c21d68-5c52-44bf-ac84-2c78a7151453 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:05.735414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-77c21d68-5c52-44bf-ac84-2c78a7151453 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:05.735414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:05.735414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:05.735414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:05.735414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:05.735414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:05.735414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:05.735414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:05.735414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:05.735414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:05.735414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:05.735414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:05.735414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:05.735414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:05.735414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:05.735414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:05.736598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:05.736598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:05.736598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:05.736598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:05.736598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:05.736598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:05.736598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:05.736598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:05.736598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:05.736598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:05.736598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:05.736598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:05.736598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:05.736598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:05.736598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:05.736598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:05.736598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:05.737508 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:05.737508 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:05.737508 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:05.737508 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 245/497] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:05.744352 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c21d68-5c52-44bf-ac84-2c78a7151453 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:05.764370 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:05.781471 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd65536-99fa-4b57-b553-d76002bf7beb None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:05.822761 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29040fd2-34c3-4f69-9f84-0d9f1d452286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:05.824083 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29040fd2-34c3-4f69-9f84-0d9f1d452286 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:05.824543 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29040fd2-34c3-4f69-9f84-0d9f1d452286 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:05.824969 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29040fd2-34c3-4f69-9f84-0d9f1d452286 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:05.825879 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29040fd2-34c3-4f69-9f84-0d9f1d452286 None admin] RBAC: Adding query filter params (role.id=8a0215180fe04d5a8f2c24b370382456, scope.project.id=932cdb9ef7644bf5ba57c6d1d6bce7ff, user.id=414707c545f641d3935720356b326672) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:05.826331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29040fd2-34c3-4f69-9f84-0d9f1d452286 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:05.827946 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29040fd2-34c3-4f69-9f84-0d9f1d452286 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:05.836024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 253/498] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Oct 17 03:33:05 2022] GET /identity/v3/role_assignments?user.id=414707c545f641d3935720356b326672&scope.project.id=932cdb9ef7644bf5ba57c6d1d6bce7ff&role.id=8a0215180fe04d5a8f2c24b370382456 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:05.946017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c21d68-5c52-44bf-ac84-2c78a7151453 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:06.004305 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5718f-0413-4c5d-a2f5-9d7fea235830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:06.004938 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5718f-0413-4c5d-a2f5-9d7fea235830 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:06.005219 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5718f-0413-4c5d-a2f5-9d7fea235830 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:06.005490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5718f-0413-4c5d-a2f5-9d7fea235830 None admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:06.006142 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5718f-0413-4c5d-a2f5-9d7fea235830 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:06.006443 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5718f-0413-4c5d-a2f5-9d7fea235830 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:06.007834 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5718f-0413-4c5d-a2f5-9d7fea235830 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:06.027363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 246/499] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:06.032890 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29040fd2-34c3-4f69-9f84-0d9f1d452286 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:06.091794 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5718f-0413-4c5d-a2f5-9d7fea235830 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:06.134358 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:06.168029 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29040fd2-34c3-4f69-9f84-0d9f1d452286 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:06.210391 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9761fe-8194-4453-a8fb-b30529d92a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:06.210848 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9761fe-8194-4453-a8fb-b30529d92a8d None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:06.211129 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9761fe-8194-4453-a8fb-b30529d92a8d None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:06.211401 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9761fe-8194-4453-a8fb-b30529d92a8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:06.219708 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5718f-0413-4c5d-a2f5-9d7fea235830 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:06.221072 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9761fe-8194-4453-a8fb-b30529d92a8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:06.223595 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9761fe-8194-4453-a8fb-b30529d92a8d None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:06.238436 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-ef9761fe-8194-4453-a8fb-b30529d92a8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:06.238436 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:06.238436 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:06.238436 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:06.238436 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:06.238436 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:06.238436 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:06.238436 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:06.238436 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:06.238436 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:06.238436 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:06.238436 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:06.238436 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:33:06.238436 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:33:06.238436 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:33:06.239630 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:33:06.239630 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:06.239630 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:06.239630 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:06.239630 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:06.239630 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:06.239630 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:06.239630 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:06.239630 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:06.239630 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:06.239630 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:06.239630 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:06.239630 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:06.239630 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:06.239630 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:06.239630 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:06.239630 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:06.241099 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:33:06.241099 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:33:06.241099 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:33:06.241099 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:33:06.241099 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:33:06.241099 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:33:06.241099 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:06.241099 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:06.241502 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 254/500] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:33:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:06.250261 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9761fe-8194-4453-a8fb-b30529d92a8d None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:06.267030 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5450fe2-c040-4cc5-9be9-56c0bcdaf6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:06.267853 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5450fe2-c040-4cc5-9be9-56c0bcdaf6fe None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:06.268193 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5450fe2-c040-4cc5-9be9-56c0bcdaf6fe None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:06.268569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5450fe2-c040-4cc5-9be9-56c0bcdaf6fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:06.270630 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:06.283833 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5450fe2-c040-4cc5-9be9-56c0bcdaf6fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:33:06.287766 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-a5450fe2-c040-4cc5-9be9-56c0bcdaf6fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:33:06.287766 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:06.287766 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:06.287766 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:06.287766 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:06.287766 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:06.287766 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:06.287766 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:06.287766 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:06.287766 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:06.287766 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:06.287766 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:06.287766 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:33:06.287766 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:33:06.287766 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:33:06.287766 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:33:06.288854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:33:06.288854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:33:06.288854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:33:06.288854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:33:06.288854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:06.288854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:06.288854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:06.288854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:06.288854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:06.288854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:06.288854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:06.288854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:06.288854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:06.288854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:06.288854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:06.288854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:06.289691 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:06.289691 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:06.289691 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:06.289691 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:06.289691 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:33:06.289691 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:33:06.289691 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:33:06.289691 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:33:06.289691 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:33:06.289691 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:33:06.289691 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:33:06.289691 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:06.292559 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 247/501] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Oct 17 03:33:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:33:06.299332 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5450fe2-c040-4cc5-9be9-56c0bcdaf6fe None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:06.407589 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:06.414996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5450fe2-c040-4cc5-9be9-56c0bcdaf6fe None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:06.457256 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a011e-b9cd-400b-a285-98ddd4926f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:06.457875 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a011e-b9cd-400b-a285-98ddd4926f5c None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:06.458157 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a011e-b9cd-400b-a285-98ddd4926f5c None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:06.458430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a011e-b9cd-400b-a285-98ddd4926f5c None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:06.459097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066a011e-b9cd-400b-a285-98ddd4926f5c None admin] RBAC: Adding query filter params (name=service) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:06.459402 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066a011e-b9cd-400b-a285-98ddd4926f5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:06.460822 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066a011e-b9cd-400b-a285-98ddd4926f5c None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:06.472566 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 248/502] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Oct 17 03:33:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:06.479145 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9761fe-8194-4453-a8fb-b30529d92a8d None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:06.493133 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a011e-b9cd-400b-a285-98ddd4926f5c None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:06.517890 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e757a3-e2a1-4221-8be3-4f6196dc9722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:06.518560 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e757a3-e2a1-4221-8be3-4f6196dc9722 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:06.518774 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e757a3-e2a1-4221-8be3-4f6196dc9722 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:06.519089 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e757a3-e2a1-4221-8be3-4f6196dc9722 None admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:06.519801 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e757a3-e2a1-4221-8be3-4f6196dc9722 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:06.520107 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e757a3-e2a1-4221-8be3-4f6196dc9722 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:06.521540 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e757a3-e2a1-4221-8be3-4f6196dc9722 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:06.534875 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 255/503] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Oct 17 03:33:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:06.544824 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e757a3-e2a1-4221-8be3-4f6196dc9722 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:06.680699 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a011e-b9cd-400b-a285-98ddd4926f5c None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:06.684512 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:06.743246 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e757a3-e2a1-4221-8be3-4f6196dc9722 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:06.751146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5a1af4-e058-4ba7-8690-1153c588f71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:06.752172 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a1af4-e058-4ba7-8690-1153c588f71f None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:06.752630 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a1af4-e058-4ba7-8690-1153c588f71f None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:06.753114 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a1af4-e058-4ba7-8690-1153c588f71f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:06.771158 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5a1af4-e058-4ba7-8690-1153c588f71f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:06.772925 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5a1af4-e058-4ba7-8690-1153c588f71f None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:06.801044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-bf5a1af4-e058-4ba7-8690-1153c588f71f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:06.801044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:06.801044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:06.801044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:06.801044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:06.801044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:06.801044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:06.801044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:06.801044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:06.801044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:06.801044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:06.801044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:06.801044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:06.801044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:06.801044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:06.801044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:06.802013 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:06.802013 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:06.802013 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:06.802013 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:06.802013 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:06.802013 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:06.802013 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:06.802013 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:06.802013 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:06.802013 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:06.802013 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:06.802013 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:06.802013 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:06.802013 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:06.802013 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:06.802013 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:06.802013 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:06.802967 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:06.802967 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:06.802967 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:06.815761 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 249/504] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:33:06.818232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b482b-ecfd-4ce0-bba8-764d373b2b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:06.819278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b482b-ecfd-4ce0-bba8-764d373b2b8b None admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:06.819706 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b482b-ecfd-4ce0-bba8-764d373b2b8b None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:06.820174 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77b482b-ecfd-4ce0-bba8-764d373b2b8b None admin] PATH_INFO: `/v3/projects/932cdb9ef7644bf5ba57c6d1d6bce7ff/users/d961b751763a4e49b5d4ca4c795c5bf6/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:06.822766 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b482b-ecfd-4ce0-bba8-764d373b2b8b None admin] RBAC: Authorizing `identity:create_grant(project_id=932cdb9ef7644bf5ba57c6d1d6bce7ff, user_id=d961b751763a4e49b5d4ca4c795c5bf6, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:06.824370 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77b482b-ecfd-4ce0-bba8-764d373b2b8b None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:06.829227 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5a1af4-e058-4ba7-8690-1153c588f71f None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:06.844521 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:06.932447 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 256/505] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Oct 17 03:33:06 2022] PUT /identity/v3/projects/932cdb9ef7644bf5ba57c6d1d6bce7ff/users/d961b751763a4e49b5d4ca4c795c5bf6/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:33:06.950744 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5a1af4-e058-4ba7-8690-1153c588f71f None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:06.997437 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d505cd9-cdda-4b80-a1a6-d9e0f4e37fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:06.998472 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d505cd9-cdda-4b80-a1a6-d9e0f4e37fea None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:06.998931 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d505cd9-cdda-4b80-a1a6-d9e0f4e37fea None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:06.999487 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d505cd9-cdda-4b80-a1a6-d9e0f4e37fea None admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:07.001342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d505cd9-cdda-4b80-a1a6-d9e0f4e37fea None admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:07.001907 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d505cd9-cdda-4b80-a1a6-d9e0f4e37fea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:07.003629 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d505cd9-cdda-4b80-a1a6-d9e0f4e37fea None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:07.029015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 250/506] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:07.035192 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b482b-ecfd-4ce0-bba8-764d373b2b8b None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:07.163237 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77b482b-ecfd-4ce0-bba8-764d373b2b8b None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:07.192586 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:07.199670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6260b8-0e91-4067-a2e6-18ffa3a38a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:07.206796 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6260b8-0e91-4067-a2e6-18ffa3a38a26 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:07.206796 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6260b8-0e91-4067-a2e6-18ffa3a38a26 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:07.206796 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6260b8-0e91-4067-a2e6-18ffa3a38a26 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:07.208378 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6260b8-0e91-4067-a2e6-18ffa3a38a26 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:07.209834 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6260b8-0e91-4067-a2e6-18ffa3a38a26 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:07.223159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-3b6260b8-0e91-4067-a2e6-18ffa3a38a26 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 17 03:33:07.223159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:07.223159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:07.223159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:07.223159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:07.223159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:07.223159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:07.223159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:07.223159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:07.223159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:07.223159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:07.223159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:07.223159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:33:07.223159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:33:07.223159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:33:07.223159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:33:07.224232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:07.224232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:07.224232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:33:07.224232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:07.224232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:33:07.224232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:07.224232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:07.224232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:07.224232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:07.224232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:07.224232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:07.224232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:07.224232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:07.224232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:07.224232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:07.224232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:07.224232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:07.224232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:07.225157 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:07.225157 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:07.225157 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:33:07.225157 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:33:07.225157 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:33:07.225157 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:33:07.225157 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:33:07.225157 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:33:07.225157 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 17 03:33:07.225157 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:07.226226 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 257/507] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Oct 17 03:33:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:07.232472 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d505cd9-cdda-4b80-a1a6-d9e0f4e37fea None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:07.345283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:07.356701 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb63cf-d6d9-4880-99ba-aec110fc8730 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:07.357121 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb63cf-d6d9-4880-99ba-aec110fc8730 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:07.357494 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefb63cf-d6d9-4880-99ba-aec110fc8730 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:07.359149 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 258/508] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:07.365073 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c007593-e6ba-4ae6-94c2-facbbacf3d38 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:07.365279 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c007593-e6ba-4ae6-94c2-facbbacf3d38 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:07.365531 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c007593-e6ba-4ae6-94c2-facbbacf3d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:07.386064 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d505cd9-cdda-4b80-a1a6-d9e0f4e37fea None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:07.402308 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-3c007593-e6ba-4ae6-94c2-facbbacf3d38 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:07.403612 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-3c007593-e6ba-4ae6-94c2-facbbacf3d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:07.422751 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5252de8-4e9d-4e75-a763-28be3b3b9202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:07.424198 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5252de8-4e9d-4e75-a763-28be3b3b9202 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:07.424476 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5252de8-4e9d-4e75-a763-28be3b3b9202 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:07.424752 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5252de8-4e9d-4e75-a763-28be3b3b9202 None admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:07.425482 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5252de8-4e9d-4e75-a763-28be3b3b9202 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:07.425787 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5252de8-4e9d-4e75-a763-28be3b3b9202 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:07.427214 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5252de8-4e9d-4e75-a763-28be3b3b9202 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:07.447567 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 251/509] 10.222.0.17 () {56 vars in 1135 bytes} [Mon Oct 17 03:33:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:07.448855 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e982d7-953b-4b1a-b906-9fe093dff0b7 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:07.449141 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e982d7-953b-4b1a-b906-9fe093dff0b7 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:07.449415 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e982d7-953b-4b1a-b906-9fe093dff0b7 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:07.450778 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 252/510] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:07.457747 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfd4d0-fb4e-42f0-aebd-79f4ff8d6e87 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:07.458034 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfd4d0-fb4e-42f0-aebd-79f4ff8d6e87 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:07.458309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbfd4d0-fb4e-42f0-aebd-79f4ff8d6e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:07.483014 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-9cbfd4d0-fb4e-42f0-aebd-79f4ff8d6e87 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:07.484431 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-9cbfd4d0-fb4e-42f0-aebd-79f4ff8d6e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:07.497236 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 259/511] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:33:07.498347 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c007593-e6ba-4ae6-94c2-facbbacf3d38 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:07.580950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 253/512] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:33:07.635861 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c007593-e6ba-4ae6-94c2-facbbacf3d38 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:07.657451 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:07.688453 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65705208-ef06-481f-8bc4-633324cb585c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:07.689139 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65705208-ef06-481f-8bc4-633324cb585c None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:07.689433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65705208-ef06-481f-8bc4-633324cb585c None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:07.689713 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65705208-ef06-481f-8bc4-633324cb585c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:07.789791 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65705208-ef06-481f-8bc4-633324cb585c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:07.791780 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65705208-ef06-481f-8bc4-633324cb585c None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:07.829717 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbfd4d0-fb4e-42f0-aebd-79f4ff8d6e87 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:07.907787 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 260/513] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:33:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 17 03:33:07.912630 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65705208-ef06-481f-8bc4-633324cb585c None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:07.931418 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:07.938044 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:07.990901 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbfd4d0-fb4e-42f0-aebd-79f4ff8d6e87 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:08.029392 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65705208-ef06-481f-8bc4-633324cb585c None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:08.036543 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1210b1-c33a-472c-b28e-9340761d6fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:08.037308 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1210b1-c33a-472c-b28e-9340761d6fbb None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:08.037695 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1210b1-c33a-472c-b28e-9340761d6fbb None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:08.038049 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1210b1-c33a-472c-b28e-9340761d6fbb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:08.038709 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1210b1-c33a-472c-b28e-9340761d6fbb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:08.041313 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1210b1-c33a-472c-b28e-9340761d6fbb None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:08.056384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-ca1210b1-c33a-472c-b28e-9340761d6fbb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 03:33:08.056384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:08.056384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:33:08.056384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:33:08.056384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:33:08.056384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:33:08.056384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:33:08.056384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:08.056384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 03:33:08.056384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:08.056384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:33:08.056384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:08.056384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:08.056384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:08.056384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:08.056384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:08.056384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:08.057421 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:08.057421 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:08.057421 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:08.057421 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:08.057421 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:08.057421 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:08.057421 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:33:08.057421 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:33:08.057421 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:33:08.057421 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:33:08.057421 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:08.057421 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:08.057421 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:08.057421 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:08.057421 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:08.057421 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:08.058249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:08.058249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:08.058249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:08.058249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:08.058249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:08.058249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:08.058249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:08.058249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:08.058249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:08.058249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:08.058249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:33:08.058249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:08.058249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 03:33:08.058249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:08.058249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 254/514] 10.222.0.17 () {56 vars in 1129 bytes} [Mon Oct 17 03:33:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:33:08.059018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1210b1-c33a-472c-b28e-9340761d6fbb None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:08.078709 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ad8b3-f65b-4602-9cd8-ccbf6ab28cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:08.079441 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ad8b3-f65b-4602-9cd8-ccbf6ab28cc0 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:08.079929 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ad8b3-f65b-4602-9cd8-ccbf6ab28cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:08.080278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ad8b3-f65b-4602-9cd8-ccbf6ab28cc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:08.093454 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ad8b3-f65b-4602-9cd8-ccbf6ab28cc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:08.095673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7ad8b3-f65b-4602-9cd8-ccbf6ab28cc0 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:08.114108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-3d7ad8b3-f65b-4602-9cd8-ccbf6ab28cc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:08.114108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:08.114108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:08.114108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:08.114108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:08.114108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:08.114108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:08.114108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:08.114108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:08.114108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:08.114108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:08.114108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:08.114108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:08.114108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:08.114108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:08.114108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:08.115078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:08.115078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:08.115078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:08.115078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:08.115078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:08.115078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:08.115078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:08.115078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:08.115078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:08.115078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:08.115078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:08.115078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:08.115078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:08.115078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:08.115078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:08.115078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:08.115078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:08.116108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:08.116108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:08.116108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:08.116108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 261/515] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:08.116108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ad8b3-f65b-4602-9cd8-ccbf6ab28cc0 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:08.186965 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1210b1-c33a-472c-b28e-9340761d6fbb None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:08.202574 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:08.219687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:08.237350 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ad8b3-f65b-4602-9cd8-ccbf6ab28cc0 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:08.239309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b5dca-5177-4c2e-97b8-cdf5161cb099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:08.240963 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b5dca-5177-4c2e-97b8-cdf5161cb099 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:08.241439 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b5dca-5177-4c2e-97b8-cdf5161cb099 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:08.241914 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b5dca-5177-4c2e-97b8-cdf5161cb099 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:08.256257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b5dca-5177-4c2e-97b8-cdf5161cb099 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:08.258346 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b5dca-5177-4c2e-97b8-cdf5161cb099 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:08.277064 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73603e5-fb89-47a1-ab4f-f0920ac9ed94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:08.277762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73603e5-fb89-47a1-ab4f-f0920ac9ed94 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:08.278113 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73603e5-fb89-47a1-ab4f-f0920ac9ed94 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:08.278491 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73603e5-fb89-47a1-ab4f-f0920ac9ed94 None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:08.279195 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73603e5-fb89-47a1-ab4f-f0920ac9ed94 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:08.279565 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73603e5-fb89-47a1-ab4f-f0920ac9ed94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:08.281027 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73603e5-fb89-47a1-ab4f-f0920ac9ed94 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:08.281484 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-238b5dca-5177-4c2e-97b8-cdf5161cb099 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:08.281484 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:08.281484 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:08.281484 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:08.281484 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:08.281484 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:08.281484 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:08.281484 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:08.281484 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:08.281484 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:08.281484 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:08.281484 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:08.281484 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:08.281484 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:08.281484 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:08.281484 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:08.282601 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:08.282601 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:08.282601 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:08.282601 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:08.282601 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:08.282601 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:08.282601 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:08.282601 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:08.282601 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:08.282601 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:08.282601 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:08.282601 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:08.282601 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:08.282601 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:08.282601 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:08.282601 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:08.282601 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:08.283695 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:08.283695 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:08.283695 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:08.290308 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 255/516] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:08.290308 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b5dca-5177-4c2e-97b8-cdf5161cb099 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:08.297412 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 262/517] 10.222.0.17 () {56 vars in 1124 bytes} [Mon Oct 17 03:33:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:08.298359 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73603e5-fb89-47a1-ab4f-f0920ac9ed94 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:08.412907 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b5dca-5177-4c2e-97b8-cdf5161cb099 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:08.416210 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73603e5-fb89-47a1-ab4f-f0920ac9ed94 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:08.455843 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e232c8-b19a-417d-8c3a-11fde5287a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:08.456425 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e232c8-b19a-417d-8c3a-11fde5287a85 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:08.456708 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e232c8-b19a-417d-8c3a-11fde5287a85 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:08.456981 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e232c8-b19a-417d-8c3a-11fde5287a85 None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:08.457638 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e232c8-b19a-417d-8c3a-11fde5287a85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:08.457944 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e232c8-b19a-417d-8c3a-11fde5287a85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:08.458689 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708386b3-207a-4e73-8706-4857c9dc4e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:08.459397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e232c8-b19a-417d-8c3a-11fde5287a85 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:08.460168 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708386b3-207a-4e73-8706-4857c9dc4e74 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:08.460677 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708386b3-207a-4e73-8706-4857c9dc4e74 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:08.461412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708386b3-207a-4e73-8706-4857c9dc4e74 None admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:08.462304 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:08.463269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708386b3-207a-4e73-8706-4857c9dc4e74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:08.464236 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708386b3-207a-4e73-8706-4857c9dc4e74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:08.466134 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708386b3-207a-4e73-8706-4857c9dc4e74 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:08.484184 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 263/518] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:08.484917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e232c8-b19a-417d-8c3a-11fde5287a85 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:08.488521 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 256/519] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:08.493866 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708386b3-207a-4e73-8706-4857c9dc4e74 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:08.529100 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:08.608603 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708386b3-207a-4e73-8706-4857c9dc4e74 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:08.611881 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e232c8-b19a-417d-8c3a-11fde5287a85 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:08.644772 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6f548-cf7d-4f03-bfa5-719282741248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:08.645407 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6f548-cf7d-4f03-bfa5-719282741248 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:08.645689 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6f548-cf7d-4f03-bfa5-719282741248 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:08.647953 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6f548-cf7d-4f03-bfa5-719282741248 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:08.648060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e178a-0b23-4b64-b474-f0b76ea8465f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:08.648636 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e178a-0b23-4b64-b474-f0b76ea8465f None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:08.648917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e178a-0b23-4b64-b474-f0b76ea8465f None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:08.649194 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e178a-0b23-4b64-b474-f0b76ea8465f None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:08.649837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e178a-0b23-4b64-b474-f0b76ea8465f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:08.650125 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e178a-0b23-4b64-b474-f0b76ea8465f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:08.651475 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e178a-0b23-4b64-b474-f0b76ea8465f None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:08.654003 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6f548-cf7d-4f03-bfa5-719282741248 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:08.655643 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6f548-cf7d-4f03-bfa5-719282741248 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:08.657053 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 264/520] 10.222.0.17 () {56 vars in 1124 bytes} [Mon Oct 17 03:33:08 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:08.657818 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e178a-0b23-4b64-b474-f0b76ea8465f None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:08.676047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-20d6f548-cf7d-4f03-bfa5-719282741248 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:08.676047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:08.676047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:08.676047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:08.676047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:08.676047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:08.676047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:08.676047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:08.676047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:08.676047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:08.676047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:08.676047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:08.676047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:33:08.676047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:33:08.676047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:33:08.676996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:33:08.676996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:08.676996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:08.676996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:08.676996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:08.676996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:08.676996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:08.676996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:08.676996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:08.676996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:08.676996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:08.676996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:08.676996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:08.676996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:08.676996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:08.676996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:08.676996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:08.677878 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:33:08.677878 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:33:08.677878 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:33:08.677878 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:33:08.677878 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:33:08.677878 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:33:08.677878 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:08.677878 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:08.680221 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 257/521] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:33:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:08.682276 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02356e90-85e6-4dbb-a41a-f36aa13be73d None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:08.682728 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02356e90-85e6-4dbb-a41a-f36aa13be73d None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:08.687869 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02356e90-85e6-4dbb-a41a-f36aa13be73d None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:08.687869 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 258/522] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:08.690448 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4b49a-ef4b-454a-950c-03fda5e6e5cf None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:08.690745 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4b49a-ef4b-454a-950c-03fda5e6e5cf None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:08.691065 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4b49a-ef4b-454a-950c-03fda5e6e5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:08.712960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-20d4b49a-ef4b-454a-950c-03fda5e6e5cf None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:08.714768 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-20d4b49a-ef4b-454a-950c-03fda5e6e5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:08.740348 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:08.795765 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 259/523] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 03:33:08.795765 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4b49a-ef4b-454a-950c-03fda5e6e5cf None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:08.797352 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e178a-0b23-4b64-b474-f0b76ea8465f None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:08.832124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:08.871718 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440e921-a478-4b40-8254-bf5edb2b0ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:08.872560 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440e921-a478-4b40-8254-bf5edb2b0ec3 None admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:08.872850 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440e921-a478-4b40-8254-bf5edb2b0ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:08.873135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440e921-a478-4b40-8254-bf5edb2b0ec3 None admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:08.873885 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9440e921-a478-4b40-8254-bf5edb2b0ec3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:08.875400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9440e921-a478-4b40-8254-bf5edb2b0ec3 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:08.934595 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4b49a-ef4b-454a-950c-03fda5e6e5cf None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:08.951956 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 265/524] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Oct 17 03:33:08 2022] POST /identity/v3/users => generated 254 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:33:08.979946 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f730d7-c663-4675-86e0-075833118ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:08.980918 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f730d7-c663-4675-86e0-075833118ac6 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:08.981383 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f730d7-c663-4675-86e0-075833118ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:08.981800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f730d7-c663-4675-86e0-075833118ac6 None admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:08.982672 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f730d7-c663-4675-86e0-075833118ac6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:08.983132 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f730d7-c663-4675-86e0-075833118ac6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:08.984809 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f730d7-c663-4675-86e0-075833118ac6 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:08.999662 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 260/525] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Oct 17 03:33:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:09.007919 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440e921-a478-4b40-8254-bf5edb2b0ec3 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:09.124078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440e921-a478-4b40-8254-bf5edb2b0ec3 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:09.141393 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:09.161022 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951d30e-d305-4454-9f40-125b384fc8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:09.162827 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951d30e-d305-4454-9f40-125b384fc8e5 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:09.162827 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951d30e-d305-4454-9f40-125b384fc8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:09.162827 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951d30e-d305-4454-9f40-125b384fc8e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:09.163038 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951d30e-d305-4454-9f40-125b384fc8e5 None admin] RBAC: Adding query filter params (role.id=8a0215180fe04d5a8f2c24b370382456, scope.project.id=932cdb9ef7644bf5ba57c6d1d6bce7ff, user.id=5f64aaf4d408462a88963973d42abfb9) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:09.163224 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951d30e-d305-4454-9f40-125b384fc8e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:09.164703 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951d30e-d305-4454-9f40-125b384fc8e5 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:09.175864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 266/526] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Oct 17 03:33:09 2022] GET /identity/v3/role_assignments?user.id=5f64aaf4d408462a88963973d42abfb9&scope.project.id=932cdb9ef7644bf5ba57c6d1d6bce7ff&role.id=8a0215180fe04d5a8f2c24b370382456 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:09.212632 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f730d7-c663-4675-86e0-075833118ac6 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:09.343096 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:09.367335 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f730d7-c663-4675-86e0-075833118ac6 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:09.413516 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0867368-0876-4aa3-9010-ac424e3e7749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:09.414319 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0867368-0876-4aa3-9010-ac424e3e7749 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:09.414603 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0867368-0876-4aa3-9010-ac424e3e7749 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:09.414879 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0867368-0876-4aa3-9010-ac424e3e7749 None admin] PATH_INFO: `/v3/services/network` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:09.416083 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0867368-0876-4aa3-9010-ac424e3e7749 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:09.417642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0867368-0876-4aa3-9010-ac424e3e7749 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:09.429929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-f0867368-0876-4aa3-9010-ac424e3e7749 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 03:33:09.429929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:09.429929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:33:09.429929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:33:09.429929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:33:09.429929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:33:09.429929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:33:09.429929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:09.429929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 03:33:09.429929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:09.429929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:33:09.429929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:09.429929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:09.429929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:09.429929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:09.429929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:09.429929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:09.431130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:09.431130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:09.431130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:09.431130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:09.431130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:09.431130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:09.431130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:33:09.431130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:33:09.431130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:33:09.431130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:33:09.431130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:09.431130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:09.431130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:09.431130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:09.431130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:09.431130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:09.432868 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:09.432868 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:09.432868 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:09.432868 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:09.432868 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:09.432868 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:09.432868 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:09.432868 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:09.432868 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:09.432868 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:09.432868 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:33:09.432868 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:09.432868 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 03:33:09.432868 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:09.432868 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 261/527] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:33:09 2022] GET /identity/v3/services/network => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:09.437686 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951d30e-d305-4454-9f40-125b384fc8e5 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:09.542659 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8731e-0283-41a7-800f-f842c9b234ef None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:09.543195 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8731e-0283-41a7-800f-f842c9b234ef None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:09.543195 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8731e-0283-41a7-800f-f842c9b234ef None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:09.544711 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 262/528] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:33:09.550219 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951d30e-d305-4454-9f40-125b384fc8e5 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:09.550590 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f6f40-0e83-4d28-86ab-9dd33ad98379 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:09.550874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f6f40-0e83-4d28-86ab-9dd33ad98379 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:09.551143 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426f6f40-0e83-4d28-86ab-9dd33ad98379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:09.586385 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-426f6f40-0e83-4d28-86ab-9dd33ad98379 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:09.588307 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-426f6f40-0e83-4d28-86ab-9dd33ad98379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:09.597698 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86131d-a331-4fb8-8a13-72db2297199c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:09.598245 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86131d-a331-4fb8-8a13-72db2297199c None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:09.598523 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86131d-a331-4fb8-8a13-72db2297199c None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:09.598795 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86131d-a331-4fb8-8a13-72db2297199c None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:09.599435 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86131d-a331-4fb8-8a13-72db2297199c None admin] RBAC: Adding query filter params (name=network) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:09.599755 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86131d-a331-4fb8-8a13-72db2297199c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:09.601118 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86131d-a331-4fb8-8a13-72db2297199c None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:09.609531 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 267/529] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Oct 17 03:33:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:09.614945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86131d-a331-4fb8-8a13-72db2297199c None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:09.707116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:09.757953 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 263/530] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 03:33:09.763323 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86131d-a331-4fb8-8a13-72db2297199c None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:09.799884 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641386b-2cf3-4ec4-8caf-c6c5b5a6a3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:09.800499 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641386b-2cf3-4ec4-8caf-c6c5b5a6a3d6 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:09.800779 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641386b-2cf3-4ec4-8caf-c6c5b5a6a3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:09.801051 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8641386b-2cf3-4ec4-8caf-c6c5b5a6a3d6 None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:09.801680 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8641386b-2cf3-4ec4-8caf-c6c5b5a6a3d6 None admin] RBAC: Adding query filter params (type=network) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:09.801981 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8641386b-2cf3-4ec4-8caf-c6c5b5a6a3d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:09.803430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8641386b-2cf3-4ec4-8caf-c6c5b5a6a3d6 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:09.808214 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 268/531] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Oct 17 03:33:09 2022] GET /identity/v3/services?type=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:10.082567 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f6f40-0e83-4d28-86ab-9dd33ad98379 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:10.188258 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:10.219105 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426f6f40-0e83-4d28-86ab-9dd33ad98379 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:10.269579 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498acd3-5f81-492e-b3db-97e3f030c702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:10.270219 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498acd3-5f81-492e-b3db-97e3f030c702 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:10.270501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498acd3-5f81-492e-b3db-97e3f030c702 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:10.270774 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498acd3-5f81-492e-b3db-97e3f030c702 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:10.291058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0498acd3-5f81-492e-b3db-97e3f030c702 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:33:10.296279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-0498acd3-5f81-492e-b3db-97e3f030c702 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:33:10.296279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:10.296279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:10.296279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:10.296279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:10.296279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:10.296279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:10.296279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:10.296279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:10.296279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:10.296279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:10.296279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:10.296279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:33:10.296279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:33:10.296279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:33:10.296279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:33:10.297273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:33:10.297273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:33:10.297273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:33:10.297273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:33:10.297273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:10.297273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:10.297273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:10.297273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:10.297273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:10.297273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:10.297273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:10.297273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:10.297273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:10.297273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:10.297273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:10.297273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:10.298084 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:10.298084 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:10.298084 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:10.298084 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:10.298084 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:33:10.298084 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:33:10.298084 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:33:10.298084 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:33:10.298084 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:33:10.298084 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:33:10.298084 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:33:10.298084 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:10.298084 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 264/532] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Oct 17 03:33:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:10.304998 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641386b-2cf3-4ec4-8caf-c6c5b5a6a3d6 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:10.454664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8641386b-2cf3-4ec4-8caf-c6c5b5a6a3d6 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:10.465164 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:10.502294 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1832d9-262d-49b1-b5ba-4e0f49439579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:10.503290 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1832d9-262d-49b1-b5ba-4e0f49439579 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:10.503788 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1832d9-262d-49b1-b5ba-4e0f49439579 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:10.504256 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1832d9-262d-49b1-b5ba-4e0f49439579 None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:10.505124 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1832d9-262d-49b1-b5ba-4e0f49439579 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:10.505627 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1832d9-262d-49b1-b5ba-4e0f49439579 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:10.507241 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1832d9-262d-49b1-b5ba-4e0f49439579 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:10.518420 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 269/533] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:33:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:10.524429 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498acd3-5f81-492e-b3db-97e3f030c702 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:10.654385 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498acd3-5f81-492e-b3db-97e3f030c702 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:10.692618 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b1e75-43d6-46ef-8fa9-df6621f254fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:10.693254 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b1e75-43d6-46ef-8fa9-df6621f254fd None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:10.693532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b1e75-43d6-46ef-8fa9-df6621f254fd None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:10.693803 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9b1e75-43d6-46ef-8fa9-df6621f254fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:10.697106 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:10.705851 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b1e75-43d6-46ef-8fa9-df6621f254fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:10.707318 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9b1e75-43d6-46ef-8fa9-df6621f254fd None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:10.723275 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-eb9b1e75-43d6-46ef-8fa9-df6621f254fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:10.723275 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:10.723275 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:10.723275 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:10.723275 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:10.723275 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:10.723275 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:10.723275 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:10.723275 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:10.723275 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:10.723275 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:10.723275 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:10.723275 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:10.723275 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:10.723275 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:10.723275 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:10.726325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:10.726325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:10.726325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:10.726325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:10.726325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:10.726325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:10.726325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:10.726325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:10.726325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:10.726325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:10.726325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:10.726325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:10.726325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:10.726325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:10.726325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:10.726325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:10.726325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:10.727193 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:10.727193 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:10.727193 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:10.727193 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 265/534] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:10.730230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1832d9-262d-49b1-b5ba-4e0f49439579 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:10.831293 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1832d9-262d-49b1-b5ba-4e0f49439579 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:10.865619 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ba756-db7f-40db-bafc-376912f903a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:10.866227 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ba756-db7f-40db-bafc-376912f903a5 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:10.866503 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ba756-db7f-40db-bafc-376912f903a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:10.866771 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ba756-db7f-40db-bafc-376912f903a5 None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:10.867406 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ba756-db7f-40db-bafc-376912f903a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:10.867707 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ba756-db7f-40db-bafc-376912f903a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:10.869609 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ba756-db7f-40db-bafc-376912f903a5 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:10.881854 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 270/535] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:10.887864 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b1e75-43d6-46ef-8fa9-df6621f254fd None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:10.990498 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9b1e75-43d6-46ef-8fa9-df6621f254fd None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:11.027433 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b294b0-52ba-4b6d-8440-0363facfa09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:11.028135 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b294b0-52ba-4b6d-8440-0363facfa09d None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:11.028422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b294b0-52ba-4b6d-8440-0363facfa09d None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:11.028697 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b294b0-52ba-4b6d-8440-0363facfa09d None admin] PATH_INFO: `/v3/users/nova` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:11.045184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b294b0-52ba-4b6d-8440-0363facfa09d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:11.046764 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b294b0-52ba-4b6d-8440-0363facfa09d None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:11.059682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-63b294b0-52ba-4b6d-8440-0363facfa09d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 03:33:11.059682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:11.059682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:11.059682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:11.059682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:11.059682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:11.059682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:11.059682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:11.059682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:11.059682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:11.059682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:11.059682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:11.059682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:33:11.059682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:33:11.059682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:33:11.059682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:33:11.060656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:11.060656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:11.060656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:33:11.060656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:11.060656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:33:11.060656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:11.060656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:11.060656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:11.060656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:11.060656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:11.060656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:11.060656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:11.060656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:11.060656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:11.060656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:11.060656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:11.060656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:11.060656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:11.061568 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:11.061568 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:11.061568 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:33:11.061568 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:33:11.061568 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:33:11.061568 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:33:11.061568 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:33:11.061568 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:33:11.061568 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 03:33:11.061568 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:11.061568 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 266/536] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Oct 17 03:33:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:11.067191 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ba756-db7f-40db-bafc-376912f903a5 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:11.108396 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:11.179865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ba756-db7f-40db-bafc-376912f903a5 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:11.209821 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1412c-6ade-4fcc-8b71-75fbcfadfae6 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:11.210158 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1412c-6ade-4fcc-8b71-75fbcfadfae6 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:11.210372 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1412c-6ade-4fcc-8b71-75fbcfadfae6 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:11.211865 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 267/537] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:33:11.217512 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36053d-f0a9-4d1c-b33b-20581f193b4a None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:11.218026 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36053d-f0a9-4d1c-b33b-20581f193b4a None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:11.218503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36053d-f0a9-4d1c-b33b-20581f193b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:11.220986 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787153e-0e42-4ebf-8389-ee3f9c2a2f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:11.221834 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787153e-0e42-4ebf-8389-ee3f9c2a2f1b None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:11.222309 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787153e-0e42-4ebf-8389-ee3f9c2a2f1b None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:11.222790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787153e-0e42-4ebf-8389-ee3f9c2a2f1b None admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:11.223791 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b787153e-0e42-4ebf-8389-ee3f9c2a2f1b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:11.224312 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b787153e-0e42-4ebf-8389-ee3f9c2a2f1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:11.225909 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b787153e-0e42-4ebf-8389-ee3f9c2a2f1b None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:11.244107 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-3b36053d-f0a9-4d1c-b33b-20581f193b4a None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:11.244107 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-3b36053d-f0a9-4d1c-b33b-20581f193b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:11.259700 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 271/538] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Oct 17 03:33:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:11.266151 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787153e-0e42-4ebf-8389-ee3f9c2a2f1b None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:11.332095 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:11.392004 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:11.401398 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 268/539] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 17 03:33:11.405063 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787153e-0e42-4ebf-8389-ee3f9c2a2f1b None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:11.453019 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcd96e-5add-41d2-ab3f-1f21fdc72205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:11.453650 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcd96e-5add-41d2-ab3f-1f21fdc72205 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:11.453933 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcd96e-5add-41d2-ab3f-1f21fdc72205 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:11.454201 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcd96e-5add-41d2-ab3f-1f21fdc72205 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:11.547848 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dcd96e-5add-41d2-ab3f-1f21fdc72205 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:11.547848 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dcd96e-5add-41d2-ab3f-1f21fdc72205 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:11.637149 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:11.651764 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 272/540] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:33:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 17 03:33:11.657341 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36053d-f0a9-4d1c-b33b-20581f193b4a None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:11.837009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36053d-f0a9-4d1c-b33b-20581f193b4a None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:11.879179 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998f9a6-c9df-437e-850a-d2b7faed9963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:11.879848 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998f9a6-c9df-437e-850a-d2b7faed9963 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:11.880133 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998f9a6-c9df-437e-850a-d2b7faed9963 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:11.880409 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998f9a6-c9df-437e-850a-d2b7faed9963 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:11.894369 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998f9a6-c9df-437e-850a-d2b7faed9963 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:11.895836 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998f9a6-c9df-437e-850a-d2b7faed9963 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:11.912126 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4a6d2-6eff-4e69-baf5-ce5c4544d38f None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:11.912618 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4a6d2-6eff-4e69-baf5-ce5c4544d38f None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:11.912716 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4a6d2-6eff-4e69-baf5-ce5c4544d38f None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:11.914020 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 273/541] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:33:11.919488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-0998f9a6-c9df-437e-850a-d2b7faed9963 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:11.919488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:11.919488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:11.919488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:11.919488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:11.919488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:11.919488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:11.919488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:11.919488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:11.919488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:11.919488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:11.919488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:11.919488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:11.919488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:11.919488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:11.919488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:11.920458 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:11.920458 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:11.920458 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:11.920458 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:11.920458 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:11.920458 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:11.920458 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:11.920458 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:11.920458 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:11.920458 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:11.920458 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:11.920458 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:11.920458 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:11.920458 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:11.920458 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:11.920458 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:11.920458 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:11.921335 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:11.921335 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:11.921335 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:11.921504 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c86bd8-69fa-4340-a1f3-cb8ddb05b12b None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:11.921504 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c86bd8-69fa-4340-a1f3-cb8ddb05b12b None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:11.921504 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c86bd8-69fa-4340-a1f3-cb8ddb05b12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:11.928867 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 269/542] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:11.934197 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998f9a6-c9df-437e-850a-d2b7faed9963 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:11.953189 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-f6c86bd8-69fa-4340-a1f3-cb8ddb05b12b None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:11.956736 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-f6c86bd8-69fa-4340-a1f3-cb8ddb05b12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:11.984302 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:12.096174 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 274/543] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:33:12.097461 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c86bd8-69fa-4340-a1f3-cb8ddb05b12b None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:12.100865 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998f9a6-c9df-437e-850a-d2b7faed9963 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:12.144536 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787a8ab-4f4a-4352-b7a7-a1b192bf64e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:12.145156 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787a8ab-4f4a-4352-b7a7-a1b192bf64e9 None admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:12.145439 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787a8ab-4f4a-4352-b7a7-a1b192bf64e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:12.145714 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787a8ab-4f4a-4352-b7a7-a1b192bf64e9 None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:12.148480 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787a8ab-4f4a-4352-b7a7-a1b192bf64e9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:12.150979 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787a8ab-4f4a-4352-b7a7-a1b192bf64e9 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:12.247809 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 270/544] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Oct 17 03:33:11 2022] POST /identity/v3/services => generated 251 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:33:12.247809 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1e74a-51e7-4af6-a413-41e0d183b5cc None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:12.247809 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1e74a-51e7-4af6-a413-41e0d183b5cc None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:12.247809 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1e74a-51e7-4af6-a413-41e0d183b5cc None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:12.251871 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 271/545] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:12.253462 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa6785-0d54-4d98-a1e6-2973b1f4db4a None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:12.253751 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa6785-0d54-4d98-a1e6-2973b1f4db4a None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:12.254028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa6785-0d54-4d98-a1e6-2973b1f4db4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:12.256685 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c86bd8-69fa-4340-a1f3-cb8ddb05b12b None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:12.292591 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:12.298621 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-4eaa6785-0d54-4d98-a1e6-2973b1f4db4a None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:12.299901 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-4eaa6785-0d54-4d98-a1e6-2973b1f4db4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:12.348928 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf00d3f-9e29-489f-8e8a-f82d4ca895bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:12.349473 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf00d3f-9e29-489f-8e8a-f82d4ca895bf None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:12.350284 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf00d3f-9e29-489f-8e8a-f82d4ca895bf None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:12.350569 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf00d3f-9e29-489f-8e8a-f82d4ca895bf None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:12.351236 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf00d3f-9e29-489f-8e8a-f82d4ca895bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:12.351616 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf00d3f-9e29-489f-8e8a-f82d4ca895bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:12.353107 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf00d3f-9e29-489f-8e8a-f82d4ca895bf None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:12.366747 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 275/546] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:12.373484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf00d3f-9e29-489f-8e8a-f82d4ca895bf None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:12.429508 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 272/547] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:12 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:33:12.430220 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa6785-0d54-4d98-a1e6-2973b1f4db4a None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:12.472289 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:12.523948 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf00d3f-9e29-489f-8e8a-f82d4ca895bf None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:12.572707 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d2af4-45c9-48a0-93ac-6e56314d142f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:12.573698 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d2af4-45c9-48a0-93ac-6e56314d142f None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:12.574043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa6785-0d54-4d98-a1e6-2973b1f4db4a None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:12.574416 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d2af4-45c9-48a0-93ac-6e56314d142f None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:12.574902 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d2af4-45c9-48a0-93ac-6e56314d142f None admin] PATH_INFO: `/v3/projects/service` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:12.584289 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d2af4-45c9-48a0-93ac-6e56314d142f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:12.586783 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d2af4-45c9-48a0-93ac-6e56314d142f None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:12.601159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-fb7d2af4-45c9-48a0-93ac-6e56314d142f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:12.601159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:12.601159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:12.601159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:12.601159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:12.601159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:12.601159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:12.601159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:12.601159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:12.601159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:12.601159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:12.601159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:12.601159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:33:12.601159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:33:12.601159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:33:12.602052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:33:12.602052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:12.602052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:12.602052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:12.602052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:12.602052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:12.602052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:12.602052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:12.602052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:12.602052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:12.602052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:12.602052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:12.602052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:12.602052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:12.602052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:12.602052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:12.602052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:12.602983 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:33:12.602983 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:33:12.602983 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:33:12.602983 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:33:12.602983 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:33:12.602983 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:33:12.602983 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:12.602983 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:12.605096 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 276/548] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:33:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:12.610473 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d2af4-45c9-48a0-93ac-6e56314d142f None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:12.616249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10defe-1aec-4b38-9ce0-07b62a3e44cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:12.617214 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10defe-1aec-4b38-9ce0-07b62a3e44cf None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:12.617727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10defe-1aec-4b38-9ce0-07b62a3e44cf None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:12.618211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10defe-1aec-4b38-9ce0-07b62a3e44cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:12.634943 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e10defe-1aec-4b38-9ce0-07b62a3e44cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:33:12.642378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-7e10defe-1aec-4b38-9ce0-07b62a3e44cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:33:12.642378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:12.642378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:12.642378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:12.642378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:12.642378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:12.642378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:12.642378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:12.642378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:12.642378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:12.642378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:12.642378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:12.642378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:33:12.642378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:33:12.642378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:33:12.642378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:33:12.647311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:33:12.647311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:33:12.647311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:33:12.647311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:33:12.647311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:12.647311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:12.647311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:12.647311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:12.647311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:12.647311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:12.647311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:12.647311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:12.647311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:12.647311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:12.647311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:12.647311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:12.648393 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:12.648393 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:12.648393 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:12.648393 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:12.648393 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:33:12.648393 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:33:12.648393 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:33:12.648393 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:33:12.648393 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:33:12.648393 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:33:12.648393 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:33:12.648393 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:12.648393 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 273/549] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Oct 17 03:33:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:12.650413 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10defe-1aec-4b38-9ce0-07b62a3e44cf None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:12.653107 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:12.752491 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d2af4-45c9-48a0-93ac-6e56314d142f None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:12.757054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:12.780187 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10defe-1aec-4b38-9ce0-07b62a3e44cf None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:12.799433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354bc218-7f82-4b85-8b1e-ad1f8d7d9035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:12.800380 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bc218-7f82-4b85-8b1e-ad1f8d7d9035 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:12.800662 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bc218-7f82-4b85-8b1e-ad1f8d7d9035 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:12.800938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bc218-7f82-4b85-8b1e-ad1f8d7d9035 None admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:12.801640 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354bc218-7f82-4b85-8b1e-ad1f8d7d9035 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:12.801949 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354bc218-7f82-4b85-8b1e-ad1f8d7d9035 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:12.803839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354bc218-7f82-4b85-8b1e-ad1f8d7d9035 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:12.817488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539fe36-4164-49ee-b2c5-009132d1706f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:12.818193 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539fe36-4164-49ee-b2c5-009132d1706f None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:12.818549 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539fe36-4164-49ee-b2c5-009132d1706f None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:12.818895 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539fe36-4164-49ee-b2c5-009132d1706f None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:12.819451 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 277/550] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Oct 17 03:33:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:12.819717 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a539fe36-4164-49ee-b2c5-009132d1706f None admin] RBAC: Adding query filter params (name=service) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:12.820127 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a539fe36-4164-49ee-b2c5-009132d1706f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:12.820322 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354bc218-7f82-4b85-8b1e-ad1f8d7d9035 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:12.821849 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a539fe36-4164-49ee-b2c5-009132d1706f None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:12.831610 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 274/551] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Oct 17 03:33:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:12.832723 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539fe36-4164-49ee-b2c5-009132d1706f None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:12.928595 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:12.936172 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354bc218-7f82-4b85-8b1e-ad1f8d7d9035 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:12.937047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539fe36-4164-49ee-b2c5-009132d1706f None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:12.972422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375173a-0ad2-4c18-b72e-277211bd1887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:12.973176 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375173a-0ad2-4c18-b72e-277211bd1887 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:12.973490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375173a-0ad2-4c18-b72e-277211bd1887 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:12.973742 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375173a-0ad2-4c18-b72e-277211bd1887 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:12.974092 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd502e9f-b34c-438f-ad15-4e5fcb6b5600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:12.974473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375173a-0ad2-4c18-b72e-277211bd1887 None admin] RBAC: Adding query filter params (role.id=611120afb3a544d0b5161da1b769a2f8, scope.project.id=932cdb9ef7644bf5ba57c6d1d6bce7ff, user.id=d961b751763a4e49b5d4ca4c795c5bf6) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:12.974725 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd502e9f-b34c-438f-ad15-4e5fcb6b5600 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:12.974823 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375173a-0ad2-4c18-b72e-277211bd1887 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:12.975004 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd502e9f-b34c-438f-ad15-4e5fcb6b5600 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:12.975275 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd502e9f-b34c-438f-ad15-4e5fcb6b5600 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:12.976296 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375173a-0ad2-4c18-b72e-277211bd1887 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:12.988048 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 275/552] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Oct 17 03:33:12 2022] GET /identity/v3/role_assignments?user.id=d961b751763a4e49b5d4ca4c795c5bf6&scope.project.id=932cdb9ef7644bf5ba57c6d1d6bce7ff&role.id=611120afb3a544d0b5161da1b769a2f8 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:12.988048 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375173a-0ad2-4c18-b72e-277211bd1887 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:12.999904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd502e9f-b34c-438f-ad15-4e5fcb6b5600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:33:13.003064 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:13.006624 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-bd502e9f-b34c-438f-ad15-4e5fcb6b5600 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:33:13.006624 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:13.006624 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:13.006624 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:13.006624 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:13.006624 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:13.006624 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:13.006624 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:13.006624 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:13.006624 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:13.006624 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:13.006624 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:13.006624 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:33:13.006624 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:33:13.006624 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:33:13.006624 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:33:13.007681 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:33:13.007681 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:33:13.007681 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:33:13.007681 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:33:13.007681 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:13.007681 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:13.007681 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:13.007681 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:13.007681 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:13.007681 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:13.007681 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:13.007681 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:13.007681 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:13.007681 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:13.007681 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:13.007681 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:13.016037 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:13.016037 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:13.016037 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:13.016037 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:13.016037 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:33:13.016037 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:33:13.016037 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:33:13.016037 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:33:13.016037 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:33:13.016037 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:33:13.016037 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:33:13.016037 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:13.016037 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 278/553] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Oct 17 03:33:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:13.016037 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfb5ec-00e1-44a4-b807-cd16ad58d737 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:13.016809 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfb5ec-00e1-44a4-b807-cd16ad58d737 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:13.016809 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfb5ec-00e1-44a4-b807-cd16ad58d737 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:13.016809 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 279/554] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:13.016809 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abfb5ec-00e1-44a4-b807-cd16ad58d737 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:13.124313 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375173a-0ad2-4c18-b72e-277211bd1887 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:13.160398 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abfb5ec-00e1-44a4-b807-cd16ad58d737 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:13.163210 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8155d3-374e-4768-8eb6-9f9ad228eff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:13.163913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8155d3-374e-4768-8eb6-9f9ad228eff1 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:13.164194 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8155d3-374e-4768-8eb6-9f9ad228eff1 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:13.164469 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8155d3-374e-4768-8eb6-9f9ad228eff1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:13.180863 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8155d3-374e-4768-8eb6-9f9ad228eff1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:13.182650 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8155d3-374e-4768-8eb6-9f9ad228eff1 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:13.198854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-0f8155d3-374e-4768-8eb6-9f9ad228eff1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:13.198854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:13.198854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:13.198854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:13.198854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:13.198854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:13.198854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:13.198854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:13.198854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:13.198854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:13.198854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:13.198854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:13.198854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:13.198854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:13.198854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:13.198854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:13.199860 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:13.199860 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:13.199860 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:13.199860 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:13.199860 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:13.199860 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:13.199860 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:13.199860 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:13.199860 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:13.199860 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:13.199860 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:13.199860 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:13.199860 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:13.199860 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:13.199860 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:13.199860 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:13.199860 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:13.200739 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:13.200739 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:13.200739 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:13.202580 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 276/555] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:13.203978 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ce3a0-16b7-4636-846b-c9c6036c2723 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:13.204458 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ce3a0-16b7-4636-846b-c9c6036c2723 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:13.204922 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ce3a0-16b7-4636-846b-c9c6036c2723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:13.205109 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec46ce5-57f8-465d-af0c-c16330ae9257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:13.205764 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec46ce5-57f8-465d-af0c-c16330ae9257 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:13.206051 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec46ce5-57f8-465d-af0c-c16330ae9257 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:13.206324 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec46ce5-57f8-465d-af0c-c16330ae9257 None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:13.207051 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec46ce5-57f8-465d-af0c-c16330ae9257 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:13.207370 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec46ce5-57f8-465d-af0c-c16330ae9257 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:13.209714 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec46ce5-57f8-465d-af0c-c16330ae9257 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:13.220530 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 280/556] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:33:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:13.221493 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec46ce5-57f8-465d-af0c-c16330ae9257 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:13.233456 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-d66ce3a0-16b7-4636-846b-c9c6036c2723 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:13.234750 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-d66ce3a0-16b7-4636-846b-c9c6036c2723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:13.241529 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:13.292560 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:13.306519 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 277/557] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:13 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 17 03:33:13.316222 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ce3a0-16b7-4636-846b-c9c6036c2723 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:13.336790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec46ce5-57f8-465d-af0c-c16330ae9257 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:13.375099 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b337ca-0336-4055-99f3-4776ac7c8fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:13.375920 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b337ca-0336-4055-99f3-4776ac7c8fd0 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:13.376303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b337ca-0336-4055-99f3-4776ac7c8fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:13.376680 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b337ca-0336-4055-99f3-4776ac7c8fd0 None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:13.377424 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b337ca-0336-4055-99f3-4776ac7c8fd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:13.377829 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b337ca-0336-4055-99f3-4776ac7c8fd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:13.379487 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b337ca-0336-4055-99f3-4776ac7c8fd0 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:13.391425 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 281/558] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:13.396859 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b337ca-0336-4055-99f3-4776ac7c8fd0 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:13.488091 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:13.508295 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b337ca-0336-4055-99f3-4776ac7c8fd0 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:13.533642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66ce3a0-16b7-4636-846b-c9c6036c2723 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:13.545727 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6d669-81a8-4ad0-9e8c-cbc03c5069b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:13.546502 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6d669-81a8-4ad0-9e8c-cbc03c5069b9 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:13.546878 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6d669-81a8-4ad0-9e8c-cbc03c5069b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:13.547248 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6d669-81a8-4ad0-9e8c-cbc03c5069b9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:13.556082 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d6d669-81a8-4ad0-9e8c-cbc03c5069b9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:13.557904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d6d669-81a8-4ad0-9e8c-cbc03c5069b9 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:13.570050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-04d6d669-81a8-4ad0-9e8c-cbc03c5069b9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 03:33:13.570050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:13.570050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:13.570050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:13.570050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:13.570050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:13.570050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:13.570050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:13.570050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:13.570050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:13.570050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:13.570050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:13.570050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:33:13.570050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:33:13.570050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:33:13.570050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:33:13.571225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:13.571225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:13.571225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:33:13.571225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:13.571225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:33:13.571225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:13.571225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:13.571225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:13.571225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:13.571225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:13.571225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:13.571225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:13.571225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:13.571225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:13.571225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:13.571225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:13.571225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:13.571225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:13.572774 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:13.572774 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:13.572774 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:33:13.572774 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:33:13.572774 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:33:13.572774 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:33:13.572774 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:33:13.572774 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:33:13.572774 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 03:33:13.572774 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:13.573576 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 282/559] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Oct 17 03:33:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:33:13.578595 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6d669-81a8-4ad0-9e8c-cbc03c5069b9 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:13.605435 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0b2cb-2f10-465d-8136-8fb4786c6370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:13.606464 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0b2cb-2f10-465d-8136-8fb4786c6370 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:13.606997 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0b2cb-2f10-465d-8136-8fb4786c6370 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:13.607572 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0b2cb-2f10-465d-8136-8fb4786c6370 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:13.624741 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0b2cb-2f10-465d-8136-8fb4786c6370 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:13.626545 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df0b2cb-2f10-465d-8136-8fb4786c6370 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:13.650727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-3df0b2cb-2f10-465d-8136-8fb4786c6370 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:13.650727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:13.650727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:13.650727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:13.650727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:13.650727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:13.650727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:13.650727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:13.650727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:13.650727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:13.650727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:13.650727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:13.650727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:13.650727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:13.650727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:13.650727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:13.653057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:13.653057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:13.653057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:13.653057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:13.653057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:13.653057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:13.653057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:13.653057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:13.653057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:13.653057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:13.653057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:13.653057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:13.653057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:13.653057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:13.653057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:13.653057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:13.653057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:13.653938 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:13.653938 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:13.653938 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:13.655004 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 278/560] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:13.660771 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0b2cb-2f10-465d-8136-8fb4786c6370 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:13.690769 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6d669-81a8-4ad0-9e8c-cbc03c5069b9 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:13.718905 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:13.778735 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df0b2cb-2f10-465d-8136-8fb4786c6370 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:13.782807 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b74721-bdda-4159-af8e-444b06b3d98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:13.783428 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b74721-bdda-4159-af8e-444b06b3d98b None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:13.783715 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b74721-bdda-4159-af8e-444b06b3d98b None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:13.784028 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b74721-bdda-4159-af8e-444b06b3d98b None admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:13.784767 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b74721-bdda-4159-af8e-444b06b3d98b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:13.785077 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b74721-bdda-4159-af8e-444b06b3d98b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:13.786499 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b74721-bdda-4159-af8e-444b06b3d98b None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:13.805989 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:13.810170 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 283/561] 10.222.0.17 () {56 vars in 1149 bytes} [Mon Oct 17 03:33:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:13.810959 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b74721-bdda-4159-af8e-444b06b3d98b None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:13.816047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc625a-2137-4db1-a72e-9f75f39b8d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:13.816657 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc625a-2137-4db1-a72e-9f75f39b8d1b None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:13.816983 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc625a-2137-4db1-a72e-9f75f39b8d1b None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:13.817259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc625a-2137-4db1-a72e-9f75f39b8d1b None admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:13.821977 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc625a-2137-4db1-a72e-9f75f39b8d1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:13.821977 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc625a-2137-4db1-a72e-9f75f39b8d1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:13.821977 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc625a-2137-4db1-a72e-9f75f39b8d1b None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:13.837642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 279/562] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:13.838535 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc625a-2137-4db1-a72e-9f75f39b8d1b None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:13.939489 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b74721-bdda-4159-af8e-444b06b3d98b None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:13.960101 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:13.964659 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc625a-2137-4db1-a72e-9f75f39b8d1b None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:13.982784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0cf5ec-aaf6-4b01-aa2d-e5e4e8f77e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:13.983720 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cf5ec-aaf6-4b01-aa2d-e5e4e8f77e24 None admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:13.985409 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cf5ec-aaf6-4b01-aa2d-e5e4e8f77e24 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:13.985409 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cf5ec-aaf6-4b01-aa2d-e5e4e8f77e24 None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:13.985409 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0cf5ec-aaf6-4b01-aa2d-e5e4e8f77e24 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:13.986917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0cf5ec-aaf6-4b01-aa2d-e5e4e8f77e24 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:14.000067 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b81fef-dcb6-4758-b9ca-3f8f40e8be6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:14.000696 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b81fef-dcb6-4758-b9ca-3f8f40e8be6d None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:14.000972 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b81fef-dcb6-4758-b9ca-3f8f40e8be6d None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:14.001243 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b81fef-dcb6-4758-b9ca-3f8f40e8be6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:14.012967 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b81fef-dcb6-4758-b9ca-3f8f40e8be6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:14.014385 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b81fef-dcb6-4758-b9ca-3f8f40e8be6d None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:14.028909 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-34b81fef-dcb6-4758-b9ca-3f8f40e8be6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:14.028909 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:14.028909 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:14.028909 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:14.028909 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:14.028909 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:14.028909 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:14.028909 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:14.028909 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:14.028909 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:14.028909 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:14.028909 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:14.028909 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:14.028909 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:14.028909 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:14.028909 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:14.029800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:14.029800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:14.029800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:14.029800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:14.029800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:14.029800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:14.029800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:14.029800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:14.029800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:14.029800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:14.029800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:14.029800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:14.029800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:14.029800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:14.029800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:14.029800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:14.029800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:14.030699 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:14.030699 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:14.030699 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:14.030699 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 280/563] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:14.035936 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b81fef-dcb6-4758-b9ca-3f8f40e8be6d None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:14.084702 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 284/564] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Oct 17 03:33:13 2022] POST /identity/v3/services => generated 240 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:33:14.084702 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0cf5ec-aaf6-4b01-aa2d-e5e4e8f77e24 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:14.207075 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:14.210185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b81fef-dcb6-4758-b9ca-3f8f40e8be6d None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:14.291060 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e3ce8-0602-404a-9e55-394758f84c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:14.291614 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e3ce8-0602-404a-9e55-394758f84c79 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:14.291914 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e3ce8-0602-404a-9e55-394758f84c79 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:14.292421 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e3ce8-0602-404a-9e55-394758f84c79 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:14.302309 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0cf5ec-aaf6-4b01-aa2d-e5e4e8f77e24 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:14.304654 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e3ce8-0602-404a-9e55-394758f84c79 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:14.307051 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e3ce8-0602-404a-9e55-394758f84c79 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:14.319634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-950e3ce8-0602-404a-9e55-394758f84c79 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 17 03:33:14.319634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:14.319634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:14.319634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:14.319634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:14.319634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:14.319634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:14.319634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:14.319634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:14.319634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:14.319634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:14.319634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:14.319634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:33:14.319634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:33:14.319634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:33:14.319634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:33:14.320622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:14.320622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:14.320622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:33:14.320622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:14.320622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:33:14.320622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:14.320622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:14.320622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:14.320622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:14.320622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:14.320622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:14.320622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:14.320622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:14.320622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:14.320622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:14.320622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:14.320622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:14.320622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:14.321583 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:14.321583 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:14.321583 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:33:14.321583 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:33:14.321583 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:33:14.321583 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:33:14.321583 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:33:14.321583 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:33:14.321583 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 17 03:33:14.321583 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:14.323844 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 281/565] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Oct 17 03:33:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:14.331114 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e3ce8-0602-404a-9e55-394758f84c79 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:14.343208 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:14.347143 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3cf5b-1ed1-4962-9aa3-c89165e23db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:14.347790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3cf5b-1ed1-4962-9aa3-c89165e23db3 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:14.348139 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3cf5b-1ed1-4962-9aa3-c89165e23db3 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:14.348417 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de3cf5b-1ed1-4962-9aa3-c89165e23db3 None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:14.349069 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3cf5b-1ed1-4962-9aa3-c89165e23db3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:14.349372 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3cf5b-1ed1-4962-9aa3-c89165e23db3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:14.350897 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de3cf5b-1ed1-4962-9aa3-c89165e23db3 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:14.366061 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 285/566] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:14.371971 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3cf5b-1ed1-4962-9aa3-c89165e23db3 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:14.457360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e3ce8-0602-404a-9e55-394758f84c79 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:14.502664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de3cf5b-1ed1-4962-9aa3-c89165e23db3 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:14.508055 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7775ed89-c6ed-461e-ba75-16861eb47e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:14.508983 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775ed89-c6ed-461e-ba75-16861eb47e49 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:14.509415 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775ed89-c6ed-461e-ba75-16861eb47e49 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:14.509838 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7775ed89-c6ed-461e-ba75-16861eb47e49 None admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:14.510781 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7775ed89-c6ed-461e-ba75-16861eb47e49 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:14.511804 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7775ed89-c6ed-461e-ba75-16861eb47e49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:14.513407 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7775ed89-c6ed-461e-ba75-16861eb47e49 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:14.523179 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:14.536455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 282/567] 10.222.0.17 () {56 vars in 1135 bytes} [Mon Oct 17 03:33:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:14.543509 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7775ed89-c6ed-461e-ba75-16861eb47e49 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:14.571460 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19909995-9f8b-42a3-994b-3da84d3f0049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:14.573077 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19909995-9f8b-42a3-994b-3da84d3f0049 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:14.573707 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19909995-9f8b-42a3-994b-3da84d3f0049 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:14.576794 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19909995-9f8b-42a3-994b-3da84d3f0049 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:14.589737 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19909995-9f8b-42a3-994b-3da84d3f0049 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:14.592462 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19909995-9f8b-42a3-994b-3da84d3f0049 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:14.617287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-19909995-9f8b-42a3-994b-3da84d3f0049 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:14.617287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:14.617287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:14.617287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:14.617287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:14.617287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:14.617287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:14.617287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:14.617287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:14.617287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:14.617287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:14.617287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:14.617287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:33:14.617287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:33:14.617287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:33:14.618474 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:33:14.618474 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:14.618474 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:14.618474 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:14.618474 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:14.618474 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:14.618474 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:14.618474 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:14.618474 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:14.618474 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:14.618474 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:14.618474 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:14.618474 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:14.618474 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:14.618474 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:14.618474 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:14.618474 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:14.619703 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:33:14.619703 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:33:14.619703 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:33:14.619703 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:33:14.619703 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:33:14.619703 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:33:14.619703 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:14.619703 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:14.621765 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 286/568] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:33:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:33:14.627328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19909995-9f8b-42a3-994b-3da84d3f0049 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:14.682325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:14.756125 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19909995-9f8b-42a3-994b-3da84d3f0049 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:14.759620 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7775ed89-c6ed-461e-ba75-16861eb47e49 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:14.799298 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b070b061-c45d-4841-87cb-5f04a2fbba55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:14.800177 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070b061-c45d-4841-87cb-5f04a2fbba55 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:14.800709 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070b061-c45d-4841-87cb-5f04a2fbba55 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:14.801134 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b070b061-c45d-4841-87cb-5f04a2fbba55 None admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:14.801274 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a703935-9635-4dcb-8e40-8f8de9286263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:14.802016 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a703935-9635-4dcb-8e40-8f8de9286263 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:14.802387 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a703935-9635-4dcb-8e40-8f8de9286263 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:14.802789 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a703935-9635-4dcb-8e40-8f8de9286263 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:14.802974 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070b061-c45d-4841-87cb-5f04a2fbba55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:14.803495 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070b061-c45d-4841-87cb-5f04a2fbba55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:14.805653 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b070b061-c45d-4841-87cb-5f04a2fbba55 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:14.814311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a703935-9635-4dcb-8e40-8f8de9286263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:14.815895 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a703935-9635-4dcb-8e40-8f8de9286263 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:14.821385 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 287/569] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Oct 17 03:33:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:14.822972 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23356664-4431-4255-8072-c6274e7011aa None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:14.823430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23356664-4431-4255-8072-c6274e7011aa None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:14.823892 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23356664-4431-4255-8072-c6274e7011aa None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:14.825711 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 288/570] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:14.826736 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23356664-4431-4255-8072-c6274e7011aa None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:14.831490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-7a703935-9635-4dcb-8e40-8f8de9286263 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:14.831490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:14.831490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:14.831490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:14.831490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:14.831490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:14.831490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:14.831490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:14.831490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:14.831490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:14.831490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:14.831490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:14.831490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:14.831490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:14.831490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:14.831490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:14.832532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:14.832532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:14.832532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:14.832532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:14.832532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:14.832532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:14.832532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:14.832532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:14.832532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:14.832532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:14.832532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:14.832532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:14.832532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:14.832532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:14.832532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:14.832532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:14.832532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:14.833691 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:14.833691 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:14.833691 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:14.833691 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 283/571] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:14.833932 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b8cb4-cd12-47ab-acea-3f96bd5b4516 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:14.834242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b8cb4-cd12-47ab-acea-3f96bd5b4516 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:14.834346 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b8cb4-cd12-47ab-acea-3f96bd5b4516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:14.858689 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-6f9b8cb4-cd12-47ab-acea-3f96bd5b4516 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:14.859979 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6f9b8cb4-cd12-47ab-acea-3f96bd5b4516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:14.872520 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:14.934817 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 284/572] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:33:14.935767 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b8cb4-cd12-47ab-acea-3f96bd5b4516 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:14.954175 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23356664-4431-4255-8072-c6274e7011aa None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:14.991524 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:14.998609 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2ef68-a68b-4cea-9fda-204a4314fc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:14.999809 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2ef68-a68b-4cea-9fda-204a4314fc79 None admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:15.000597 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2ef68-a68b-4cea-9fda-204a4314fc79 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:15.007236 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2ef68-a68b-4cea-9fda-204a4314fc79 None admin] PATH_INFO: `/v3/projects/932cdb9ef7644bf5ba57c6d1d6bce7ff/users/cecbc19355ef4570b2b05bba876a3a8b/roles/8a0215180fe04d5a8f2c24b370382456` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:15.020116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f2ef68-a68b-4cea-9fda-204a4314fc79 None admin] RBAC: Authorizing `identity:create_grant(project_id=932cdb9ef7644bf5ba57c6d1d6bce7ff, user_id=cecbc19355ef4570b2b05bba876a3a8b, role_id=8a0215180fe04d5a8f2c24b370382456)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:15.022652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f2ef68-a68b-4cea-9fda-204a4314fc79 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:15.049242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b8cb4-cd12-47ab-acea-3f96bd5b4516 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:15.086447 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b7a46-c78b-4919-9189-0cd4f1df4415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:15.088345 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b7a46-c78b-4919-9189-0cd4f1df4415 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:15.088764 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b7a46-c78b-4919-9189-0cd4f1df4415 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:15.089164 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b7a46-c78b-4919-9189-0cd4f1df4415 None admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:15.090310 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b7a46-c78b-4919-9189-0cd4f1df4415 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:15.090661 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b7a46-c78b-4919-9189-0cd4f1df4415 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:15.092252 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b7a46-c78b-4919-9189-0cd4f1df4415 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:15.106668 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 285/573] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:15.109351 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 289/574] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Oct 17 03:33:14 2022] PUT /identity/v3/projects/932cdb9ef7644bf5ba57c6d1d6bce7ff/users/cecbc19355ef4570b2b05bba876a3a8b/roles/8a0215180fe04d5a8f2c24b370382456 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:33:15.112244 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b7a46-c78b-4919-9189-0cd4f1df4415 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:15.244607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2ef68-a68b-4cea-9fda-204a4314fc79 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:15.248797 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b7a46-c78b-4919-9189-0cd4f1df4415 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:15.259198 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:15.318434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e1378-7c11-4564-8ee9-6efd2c2b580a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:15.319094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e1378-7c11-4564-8ee9-6efd2c2b580a None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:15.319378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e1378-7c11-4564-8ee9-6efd2c2b580a None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:15.319652 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e1378-7c11-4564-8ee9-6efd2c2b580a None admin] PATH_INFO: `/v3/projects/service` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:15.327335 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e1378-7c11-4564-8ee9-6efd2c2b580a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:15.328826 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822e1378-7c11-4564-8ee9-6efd2c2b580a None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:15.342215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-822e1378-7c11-4564-8ee9-6efd2c2b580a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:15.342215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:15.342215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:15.342215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:15.342215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:15.342215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:15.342215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:15.342215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:15.342215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:15.342215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:15.342215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:15.342215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:15.342215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:33:15.342215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:33:15.342215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:33:15.343146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:33:15.343146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:15.343146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:15.343146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:15.343146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:15.343146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:15.343146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:15.343146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:15.343146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:15.343146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:15.343146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:15.343146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:15.343146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:15.343146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:15.343146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:15.343146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:15.343146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:15.344750 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:33:15.344750 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:33:15.344750 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:33:15.344750 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:33:15.344750 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:33:15.344750 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:33:15.344750 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:15.344750 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:15.344750 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 286/575] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:33:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:33:15.349489 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e1378-7c11-4564-8ee9-6efd2c2b580a None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:15.380716 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:15.400221 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2ef68-a68b-4cea-9fda-204a4314fc79 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:15.457907 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50665926-9431-4d02-a2a6-2460dbe235f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:15.458736 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50665926-9431-4d02-a2a6-2460dbe235f4 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:15.459018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50665926-9431-4d02-a2a6-2460dbe235f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:15.459290 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50665926-9431-4d02-a2a6-2460dbe235f4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:15.463757 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50665926-9431-4d02-a2a6-2460dbe235f4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:15.463757 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50665926-9431-4d02-a2a6-2460dbe235f4 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:15.482043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-50665926-9431-4d02-a2a6-2460dbe235f4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 03:33:15.482043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:15.482043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:33:15.482043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:33:15.482043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:33:15.482043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:33:15.482043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:33:15.482043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:15.482043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 03:33:15.482043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:15.482043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:33:15.482043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:15.482043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:15.482043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:15.482043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:15.482043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:15.482043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:15.483097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:15.483097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:15.483097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:15.483097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:15.483097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:15.483097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:15.483097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:33:15.483097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:33:15.483097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:33:15.483097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:33:15.483097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:15.483097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:15.483097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:15.483097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:15.483097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:15.483097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:15.484072 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:15.484072 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:15.484072 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:15.484072 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:15.484072 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:15.484072 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:15.484072 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:15.484072 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:15.484072 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:15.484072 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:15.484072 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:33:15.484072 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:15.484072 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 03:33:15.484072 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:15.485458 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 290/576] 10.222.0.17 () {56 vars in 1129 bytes} [Mon Oct 17 03:33:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:15.489844 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50665926-9431-4d02-a2a6-2460dbe235f4 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:15.510025 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e1378-7c11-4564-8ee9-6efd2c2b580a None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:15.560192 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e923e97-a06b-4dcd-809b-268db8c3a1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:15.561107 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e923e97-a06b-4dcd-809b-268db8c3a1d1 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:15.561390 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e923e97-a06b-4dcd-809b-268db8c3a1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:15.561666 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e923e97-a06b-4dcd-809b-268db8c3a1d1 None admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:15.562666 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e923e97-a06b-4dcd-809b-268db8c3a1d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:15.563463 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e923e97-a06b-4dcd-809b-268db8c3a1d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:15.565187 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e923e97-a06b-4dcd-809b-268db8c3a1d1 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:15.583582 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 287/577] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Oct 17 03:33:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:15.584878 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1a6b5-c602-4e45-b846-36379fadb441 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:15.585165 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1a6b5-c602-4e45-b846-36379fadb441 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:15.585441 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1a6b5-c602-4e45-b846-36379fadb441 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:15.587456 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 288/578] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:15.591276 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1a6b5-c602-4e45-b846-36379fadb441 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:15.623036 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:15.626608 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50665926-9431-4d02-a2a6-2460dbe235f4 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:15.661496 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00591bf0-d9ce-489f-937d-7cd6a9999c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:15.662189 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00591bf0-d9ce-489f-937d-7cd6a9999c48 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:15.662534 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00591bf0-d9ce-489f-937d-7cd6a9999c48 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:15.663013 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00591bf0-d9ce-489f-937d-7cd6a9999c48 None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:15.663710 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00591bf0-d9ce-489f-937d-7cd6a9999c48 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:15.664104 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00591bf0-d9ce-489f-937d-7cd6a9999c48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:15.666074 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00591bf0-d9ce-489f-937d-7cd6a9999c48 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:15.671824 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 291/579] 10.222.0.17 () {56 vars in 1124 bytes} [Mon Oct 17 03:33:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:15.672328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca9a93-aa40-4129-8b31-7660e52f91d7 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:15.672676 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca9a93-aa40-4129-8b31-7660e52f91d7 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:15.673017 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca9a93-aa40-4129-8b31-7660e52f91d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:15.691188 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-27ca9a93-aa40-4129-8b31-7660e52f91d7 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:15.692526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-27ca9a93-aa40-4129-8b31-7660e52f91d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:15.701164 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:15.722077 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1a6b5-c602-4e45-b846-36379fadb441 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:15.762576 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66bfbc9-1bfd-4769-83ce-12e209df8858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:15.763578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bfbc9-1bfd-4769-83ce-12e209df8858 None admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:15.764036 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bfbc9-1bfd-4769-83ce-12e209df8858 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:15.764497 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66bfbc9-1bfd-4769-83ce-12e209df8858 None admin] PATH_INFO: `/v3/projects/932cdb9ef7644bf5ba57c6d1d6bce7ff/users/5f64aaf4d408462a88963973d42abfb9/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:15.766770 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66bfbc9-1bfd-4769-83ce-12e209df8858 None admin] RBAC: Authorizing `identity:create_grant(project_id=932cdb9ef7644bf5ba57c6d1d6bce7ff, user_id=5f64aaf4d408462a88963973d42abfb9, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:15.767994 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 292/580] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:33:15.768808 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66bfbc9-1bfd-4769-83ce-12e209df8858 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:15.768919 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca9a93-aa40-4129-8b31-7660e52f91d7 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:15.877369 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca9a93-aa40-4129-8b31-7660e52f91d7 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:15.880645 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 289/581] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Oct 17 03:33:15 2022] PUT /identity/v3/projects/932cdb9ef7644bf5ba57c6d1d6bce7ff/users/5f64aaf4d408462a88963973d42abfb9/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:33:15.913483 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d6e52-ff91-4f0c-b664-a613a3233c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:15.914113 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d6e52-ff91-4f0c-b664-a613a3233c91 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:15.914399 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d6e52-ff91-4f0c-b664-a613a3233c91 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:15.914669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d6e52-ff91-4f0c-b664-a613a3233c91 None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:15.915969 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d6e52-ff91-4f0c-b664-a613a3233c91 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:15.916275 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d6e52-ff91-4f0c-b664-a613a3233c91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:15.917680 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d6e52-ff91-4f0c-b664-a613a3233c91 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:15.923848 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 293/582] 10.222.0.17 () {56 vars in 1124 bytes} [Mon Oct 17 03:33:15 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:15.929968 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66bfbc9-1bfd-4769-83ce-12e209df8858 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:16.071882 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d6e52-ff91-4f0c-b664-a613a3233c91 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:16.082462 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:16.084349 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:16.097698 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66bfbc9-1bfd-4769-83ce-12e209df8858 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:16.151773 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48351cb5-d1f8-4d6a-a138-110c690fb448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:16.152316 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48351cb5-d1f8-4d6a-a138-110c690fb448 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:16.152602 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48351cb5-d1f8-4d6a-a138-110c690fb448 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:16.152877 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48351cb5-d1f8-4d6a-a138-110c690fb448 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:16.153593 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48351cb5-d1f8-4d6a-a138-110c690fb448 None admin] RBAC: Adding query filter params (interface=public, service_id=628ae04687ef46908dcbbd4c6cbab448, region_id=RegionOne) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:16.153903 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48351cb5-d1f8-4d6a-a138-110c690fb448 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:16.155444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48351cb5-d1f8-4d6a-a138-110c690fb448 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:16.167820 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 290/583] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Oct 17 03:33:15 2022] GET /identity/v3/endpoints?service_id=628ae04687ef46908dcbbd4c6cbab448&interface=public®ion_id=RegionOne => generated 194 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:16.180771 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48351cb5-d1f8-4d6a-a138-110c690fb448 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:16.224424 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d6e52-ff91-4f0c-b664-a613a3233c91 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:16.289161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ba811-5235-4738-8b66-a5874e3fe2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:16.290761 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ba811-5235-4738-8b66-a5874e3fe2c0 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:16.291314 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ba811-5235-4738-8b66-a5874e3fe2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:16.291796 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ba811-5235-4738-8b66-a5874e3fe2c0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:16.292648 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9ba811-5235-4738-8b66-a5874e3fe2c0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:16.294183 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9ba811-5235-4738-8b66-a5874e3fe2c0 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:16.309443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-da9ba811-5235-4738-8b66-a5874e3fe2c0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 03:33:16.309443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:16.309443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:33:16.309443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:33:16.309443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:33:16.309443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:33:16.309443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:33:16.309443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:16.309443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 03:33:16.309443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:16.309443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:33:16.309443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:16.309443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:16.309443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:16.309443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:16.309443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:16.310471 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:16.310471 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:16.310471 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:16.310471 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:16.310471 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:16.310471 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:16.310471 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:16.310471 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:33:16.310471 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:33:16.310471 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:33:16.310471 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:33:16.310471 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:16.310471 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:16.310471 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:16.310471 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:16.310471 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:16.310471 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:16.311415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:16.311415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:16.311415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:16.311415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:16.311415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:16.311415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:16.311415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:16.311415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:16.311415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:16.311415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:16.311415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:33:16.311415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:16.311415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 03:33:16.311415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:16.312215 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 294/584] 10.222.0.17 () {56 vars in 1132 bytes} [Mon Oct 17 03:33:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:16.320696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ba811-5235-4738-8b66-a5874e3fe2c0 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:16.350854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48351cb5-d1f8-4d6a-a138-110c690fb448 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:16.386234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0ab37-37aa-45ee-952c-2858a23cd85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:16.387296 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0ab37-37aa-45ee-952c-2858a23cd85c None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:16.387799 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0ab37-37aa-45ee-952c-2858a23cd85c None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:16.388270 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c0ab37-37aa-45ee-952c-2858a23cd85c None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:16.389098 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c0ab37-37aa-45ee-952c-2858a23cd85c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:16.390739 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c0ab37-37aa-45ee-952c-2858a23cd85c None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:16.396725 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 291/585] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Oct 17 03:33:16 2022] GET /identity/v3/services => generated 772 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:16.462451 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:16.501786 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ba811-5235-4738-8b66-a5874e3fe2c0 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:16.550348 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b23e902-e633-4728-8efa-fe5d2148a681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:16.551370 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23e902-e633-4728-8efa-fe5d2148a681 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:16.551879 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23e902-e633-4728-8efa-fe5d2148a681 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:16.552344 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b23e902-e633-4728-8efa-fe5d2148a681 None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:16.553178 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b23e902-e633-4728-8efa-fe5d2148a681 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:16.553701 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b23e902-e633-4728-8efa-fe5d2148a681 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:16.555375 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b23e902-e633-4728-8efa-fe5d2148a681 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:16.563892 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 295/586] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Oct 17 03:33:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:16.570075 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0ab37-37aa-45ee-952c-2858a23cd85c None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:16.593729 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:16.706891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c0ab37-37aa-45ee-952c-2858a23cd85c None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:16.743773 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b06882-0492-4eee-ab8d-3b7e0522b4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:16.744396 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b06882-0492-4eee-ab8d-3b7e0522b4ab None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:16.744672 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b06882-0492-4eee-ab8d-3b7e0522b4ab None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:16.744943 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b06882-0492-4eee-ab8d-3b7e0522b4ab None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:16.745566 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b06882-0492-4eee-ab8d-3b7e0522b4ab None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:16.745866 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b06882-0492-4eee-ab8d-3b7e0522b4ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:16.747241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b06882-0492-4eee-ab8d-3b7e0522b4ab None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:16.753327 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 292/587] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Oct 17 03:33:16 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:17.103144 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c9053-7150-40f5-a03a-2806f35113ff None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:17.103641 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c9053-7150-40f5-a03a-2806f35113ff None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:17.103641 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c9053-7150-40f5-a03a-2806f35113ff None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:17.105258 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 296/588] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:17.110217 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06cc60-4408-4ad7-af77-9116a81d6244 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:17.110620 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06cc60-4408-4ad7-af77-9116a81d6244 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:17.110809 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06cc60-4408-4ad7-af77-9116a81d6244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:17.135354 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-6e06cc60-4408-4ad7-af77-9116a81d6244 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:17.136649 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6e06cc60-4408-4ad7-af77-9116a81d6244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:17.223472 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 293/589] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:33:17.517758 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c9053-7150-40f5-a03a-2806f35113ff None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:17.640352 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c9053-7150-40f5-a03a-2806f35113ff None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:17.674520 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43892434-2ed3-4763-9591-2a9d93da410c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:17.675154 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43892434-2ed3-4763-9591-2a9d93da410c None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:17.675432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43892434-2ed3-4763-9591-2a9d93da410c None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:17.675704 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43892434-2ed3-4763-9591-2a9d93da410c None admin] PATH_INFO: `/v3/services/network` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:17.676308 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43892434-2ed3-4763-9591-2a9d93da410c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:17.677673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43892434-2ed3-4763-9591-2a9d93da410c None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:17.679468 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:17.689230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-43892434-2ed3-4763-9591-2a9d93da410c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 03:33:17.689230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:17.689230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:33:17.689230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:33:17.689230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:33:17.689230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:33:17.689230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:33:17.689230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:17.689230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 03:33:17.689230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:17.689230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:33:17.689230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:17.689230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:17.689230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:17.689230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:17.689230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:17.689230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:17.690249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:17.690249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:17.690249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:17.690249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:17.690249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:17.690249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:17.690249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:33:17.690249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:33:17.690249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:33:17.690249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:33:17.690249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:17.690249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:17.690249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:17.690249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:17.690249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:17.690249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:17.691476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:17.691476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:17.691476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:17.691476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:17.691476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:17.691476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:17.691476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:17.691476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:17.691476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:17.691476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:17.691476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:33:17.691476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:17.691476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 03:33:17.691476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:17.691476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 297/590] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:33:17 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:17.697085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06cc60-4408-4ad7-af77-9116a81d6244 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:17.733890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:17.761278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf08db-31e8-4799-9ba6-56544460e427 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:17.761623 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf08db-31e8-4799-9ba6-56544460e427 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:17.761838 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf08db-31e8-4799-9ba6-56544460e427 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:17.763315 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 298/591] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:33:17.773137 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269fde7-8c78-4033-9864-d79dbd2fe43f None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:17.773388 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269fde7-8c78-4033-9864-d79dbd2fe43f None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:17.773625 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269fde7-8c78-4033-9864-d79dbd2fe43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:17.803594 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06cc60-4408-4ad7-af77-9116a81d6244 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:17.816795 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-2269fde7-8c78-4033-9864-d79dbd2fe43f None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:17.818013 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-2269fde7-8c78-4033-9864-d79dbd2fe43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:17.845014 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efb88f-23be-4f16-a5ae-274dbbd99a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:17.846051 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efb88f-23be-4f16-a5ae-274dbbd99a5c None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:17.846569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efb88f-23be-4f16-a5ae-274dbbd99a5c None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:17.847062 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efb88f-23be-4f16-a5ae-274dbbd99a5c None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:17.848074 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6efb88f-23be-4f16-a5ae-274dbbd99a5c None admin] RBAC: Adding query filter params (name=network) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:17.848606 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6efb88f-23be-4f16-a5ae-274dbbd99a5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:17.855791 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6efb88f-23be-4f16-a5ae-274dbbd99a5c None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:17.872357 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 294/592] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Oct 17 03:33:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:17.877274 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efb88f-23be-4f16-a5ae-274dbbd99a5c None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:17.954085 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 299/593] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:33:18.005763 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:18.047517 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6efb88f-23be-4f16-a5ae-274dbbd99a5c None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:18.090825 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d695e83-378a-44b0-b77c-7124d5c9335f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:18.091883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d695e83-378a-44b0-b77c-7124d5c9335f None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:18.092390 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d695e83-378a-44b0-b77c-7124d5c9335f None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:18.092872 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d695e83-378a-44b0-b77c-7124d5c9335f None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:18.093701 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d695e83-378a-44b0-b77c-7124d5c9335f None admin] RBAC: Adding query filter params (type=network) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:18.094219 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d695e83-378a-44b0-b77c-7124d5c9335f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:18.095831 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d695e83-378a-44b0-b77c-7124d5c9335f None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:18.107663 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 295/594] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Oct 17 03:33:17 2022] GET /identity/v3/services?type=network => generated 350 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:33:18.113570 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269fde7-8c78-4033-9864-d79dbd2fe43f None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:18.259856 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269fde7-8c78-4033-9864-d79dbd2fe43f None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:18.276676 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d695e83-378a-44b0-b77c-7124d5c9335f None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:18.305145 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:18.325524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f3b3d-4db6-4878-9f9a-bed071c3c578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:18.326157 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f3b3d-4db6-4878-9f9a-bed071c3c578 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:18.326443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f3b3d-4db6-4878-9f9a-bed071c3c578 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:18.326712 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6f3b3d-4db6-4878-9f9a-bed071c3c578 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:18.327433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f3b3d-4db6-4878-9f9a-bed071c3c578 None admin] RBAC: Adding query filter params (interface=public, service_id=d383fd5eec484eceb1af8cd6894dbe5b, region_id=RegionOne) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:18.327770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f3b3d-4db6-4878-9f9a-bed071c3c578 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:18.329129 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6f3b3d-4db6-4878-9f9a-bed071c3c578 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:18.346926 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 300/595] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Oct 17 03:33:18 2022] GET /identity/v3/endpoints?service_id=d383fd5eec484eceb1af8cd6894dbe5b&interface=public®ion_id=RegionOne => generated 194 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:18.354312 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f3b3d-4db6-4878-9f9a-bed071c3c578 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:18.463077 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6f3b3d-4db6-4878-9f9a-bed071c3c578 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:18.490126 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d695e83-378a-44b0-b77c-7124d5c9335f None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:18.505444 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74cce83-6be0-4a4a-8a8c-6f98457365f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:18.506441 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74cce83-6be0-4a4a-8a8c-6f98457365f5 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:18.506929 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74cce83-6be0-4a4a-8a8c-6f98457365f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:18.507396 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74cce83-6be0-4a4a-8a8c-6f98457365f5 None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:18.508227 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74cce83-6be0-4a4a-8a8c-6f98457365f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:18.510335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74cce83-6be0-4a4a-8a8c-6f98457365f5 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:18.517026 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 301/596] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Oct 17 03:33:18 2022] GET /identity/v3/services => generated 772 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:33:18.575190 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9b85d-12f7-4b22-a0a0-bad40b702e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:18.575791 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9b85d-12f7-4b22-a0a0-bad40b702e15 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:18.576080 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9b85d-12f7-4b22-a0a0-bad40b702e15 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:18.576357 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f9b85d-12f7-4b22-a0a0-bad40b702e15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:18.590775 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:18.597281 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f9b85d-12f7-4b22-a0a0-bad40b702e15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:33:18.602961 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a8e1a-83e6-4834-bb53-406deed63bdb None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:18.602961 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a8e1a-83e6-4834-bb53-406deed63bdb None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:18.602961 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a8e1a-83e6-4834-bb53-406deed63bdb None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:18.602961 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 302/597] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:33:18.603239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-17f9b85d-12f7-4b22-a0a0-bad40b702e15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:33:18.603239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:18.603239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:18.603239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:18.603239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:18.603239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:18.603239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:18.603239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:18.603239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:18.603239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:18.603239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:18.603239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:18.603239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:33:18.603239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:33:18.603239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:33:18.603239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:33:18.604124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:33:18.604124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:33:18.604124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:33:18.604124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:33:18.604124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:18.604124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:18.604124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:18.604124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:18.604124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:18.604124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:18.604124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:18.604124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:18.604124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:18.604124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:18.604124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:18.604124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:18.604962 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:18.604962 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:18.604962 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:18.604962 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:18.604962 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:33:18.604962 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:33:18.604962 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:33:18.604962 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:33:18.604962 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:33:18.604962 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:33:18.604962 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:33:18.604962 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:18.604962 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 296/598] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Oct 17 03:33:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:33:18.609483 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3918d1-5458-49dc-aa67-27a6746ed9c0 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:18.609483 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3918d1-5458-49dc-aa67-27a6746ed9c0 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:18.609483 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3918d1-5458-49dc-aa67-27a6746ed9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:18.612294 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9b85d-12f7-4b22-a0a0-bad40b702e15 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:18.630969 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-0f3918d1-5458-49dc-aa67-27a6746ed9c0 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:18.633184 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-0f3918d1-5458-49dc-aa67-27a6746ed9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:18.715110 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:18.733478 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 303/599] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:33:18.778271 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f9b85d-12f7-4b22-a0a0-bad40b702e15 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:18.813184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0bd9c-3474-46cb-9c32-9f06412e242b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:18.813845 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0bd9c-3474-46cb-9c32-9f06412e242b None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:18.814132 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0bd9c-3474-46cb-9c32-9f06412e242b None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:18.814407 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c0bd9c-3474-46cb-9c32-9f06412e242b None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:18.815074 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0bd9c-3474-46cb-9c32-9f06412e242b None admin] RBAC: Adding query filter params (name=service) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:18.815375 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0bd9c-3474-46cb-9c32-9f06412e242b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:18.816847 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c0bd9c-3474-46cb-9c32-9f06412e242b None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:18.826284 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 297/600] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Oct 17 03:33:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:33:18.831971 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3918d1-5458-49dc-aa67-27a6746ed9c0 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:18.939286 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3918d1-5458-49dc-aa67-27a6746ed9c0 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:18.976223 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7a867-acfa-4fdf-9e94-ff13b8939910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:18.976859 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7a867-acfa-4fdf-9e94-ff13b8939910 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:18.977140 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7a867-acfa-4fdf-9e94-ff13b8939910 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:18.977448 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7a867-acfa-4fdf-9e94-ff13b8939910 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:18.988097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b7a867-acfa-4fdf-9e94-ff13b8939910 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:18.989549 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b7a867-acfa-4fdf-9e94-ff13b8939910 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:19.005863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-a8b7a867-acfa-4fdf-9e94-ff13b8939910 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:19.005863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:19.005863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:19.005863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:19.005863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:19.005863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:19.005863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:19.005863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:19.005863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:19.005863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:19.005863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:19.005863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:19.005863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:19.005863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:19.005863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:19.005863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:19.006840 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:19.006840 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:19.006840 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:19.006840 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:19.006840 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:19.006840 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:19.006840 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:19.006840 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:19.006840 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:19.006840 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:19.006840 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:19.006840 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:19.006840 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:19.006840 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:19.006840 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:19.006840 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:19.006840 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:19.007692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:19.007692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:19.007692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:19.007692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 304/601] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:19.013416 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0bd9c-3474-46cb-9c32-9f06412e242b None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:19.072715 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7a867-acfa-4fdf-9e94-ff13b8939910 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:19.111595 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:19.145072 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:19.196566 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c0bd9c-3474-46cb-9c32-9f06412e242b None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:19.237560 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d0a3f-9639-4bed-981b-a2a4f1e7bc03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:19.238179 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d0a3f-9639-4bed-981b-a2a4f1e7bc03 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:19.238521 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d0a3f-9639-4bed-981b-a2a4f1e7bc03 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:19.238803 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d0a3f-9639-4bed-981b-a2a4f1e7bc03 None admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:19.239450 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7d0a3f-9639-4bed-981b-a2a4f1e7bc03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:19.239765 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7d0a3f-9639-4bed-981b-a2a4f1e7bc03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:19.241442 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7d0a3f-9639-4bed-981b-a2a4f1e7bc03 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:19.262604 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 298/602] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:19.268543 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d0a3f-9639-4bed-981b-a2a4f1e7bc03 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:19.274083 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7a867-acfa-4fdf-9e94-ff13b8939910 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:19.356711 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e018d84-8ac0-4d36-9035-63809e970d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:19.357359 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e018d84-8ac0-4d36-9035-63809e970d5c None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:19.357642 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e018d84-8ac0-4d36-9035-63809e970d5c None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:19.357914 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e018d84-8ac0-4d36-9035-63809e970d5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:19.384760 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d0a3f-9639-4bed-981b-a2a4f1e7bc03 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:19.388445 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e018d84-8ac0-4d36-9035-63809e970d5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:33:19.399118 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-1e018d84-8ac0-4d36-9035-63809e970d5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:33:19.399118 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:19.399118 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:19.399118 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:19.399118 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:19.399118 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:19.399118 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:19.399118 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:19.399118 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:19.399118 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:19.399118 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:19.399118 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:19.399118 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:33:19.399118 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:33:19.399118 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:33:19.399118 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:33:19.400146 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:33:19.400146 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:33:19.400146 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:33:19.400146 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:33:19.400146 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:19.400146 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:19.400146 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:19.400146 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:19.400146 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:19.400146 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:19.400146 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:19.400146 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:19.400146 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:19.400146 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:19.400146 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:19.400146 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:19.400976 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:19.400976 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:19.400976 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:19.400976 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:19.400976 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:33:19.400976 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:33:19.400976 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:33:19.400976 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:33:19.400976 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:33:19.400976 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:33:19.400976 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:33:19.400976 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:19.400976 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 305/603] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Oct 17 03:33:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:33:19.401683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030505a-8cca-429a-abac-f3829a8a911b None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:19.401760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030505a-8cca-429a-abac-f3829a8a911b None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:19.402348 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030505a-8cca-429a-abac-f3829a8a911b None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:19.403628 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 306/604] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:19.408443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030505a-8cca-429a-abac-f3829a8a911b None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:19.417127 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:19.433482 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886f0759-2877-4853-ac5b-aa499bb085d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:19.434132 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f0759-2877-4853-ac5b-aa499bb085d7 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:19.434391 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f0759-2877-4853-ac5b-aa499bb085d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:19.434664 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886f0759-2877-4853-ac5b-aa499bb085d7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:19.441777 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f0759-2877-4853-ac5b-aa499bb085d7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:19.443205 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886f0759-2877-4853-ac5b-aa499bb085d7 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:19.456265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-886f0759-2877-4853-ac5b-aa499bb085d7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 03:33:19.456265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:19.456265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:19.456265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:19.456265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:19.456265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:19.456265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:19.456265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:19.456265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:19.456265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:19.456265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:19.456265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:19.456265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:33:19.456265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:33:19.456265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:33:19.456265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:33:19.457252 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:19.457252 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:19.457252 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:33:19.457252 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:19.457252 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:33:19.457252 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:19.457252 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:19.457252 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:19.457252 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:19.457252 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:19.457252 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:19.457252 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:19.457252 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:19.457252 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:19.457252 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:19.457252 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:19.457252 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:19.457252 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:19.462941 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:19.462941 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:19.462941 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:33:19.462941 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:33:19.462941 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:33:19.462941 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:33:19.462941 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:33:19.462941 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:33:19.462941 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 03:33:19.462941 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:19.462941 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 299/605] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Oct 17 03:33:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:19.462941 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b061d4-49ca-46b8-82e6-612c657d79b3 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:19.462941 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b061d4-49ca-46b8-82e6-612c657d79b3 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:19.463853 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b061d4-49ca-46b8-82e6-612c657d79b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:19.479072 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-44b061d4-49ca-46b8-82e6-612c657d79b3 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:19.480290 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-44b061d4-49ca-46b8-82e6-612c657d79b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:19.522211 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:19.528134 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030505a-8cca-429a-abac-f3829a8a911b None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:19.552780 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 300/606] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:33:19.553867 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492fcd1-80bf-4942-8235-7c10bc640dfb None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:19.554151 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492fcd1-80bf-4942-8235-7c10bc640dfb None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:19.554423 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492fcd1-80bf-4942-8235-7c10bc640dfb None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:19.556453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 301/607] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:19.560972 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492fcd1-80bf-4942-8235-7c10bc640dfb None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:19.594181 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf64142f-bd9a-48b6-a59d-944680e2d6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:19.595833 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64142f-bd9a-48b6-a59d-944680e2d6a6 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:19.596941 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64142f-bd9a-48b6-a59d-944680e2d6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:19.597580 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf64142f-bd9a-48b6-a59d-944680e2d6a6 None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:19.598754 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf64142f-bd9a-48b6-a59d-944680e2d6a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:19.599259 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf64142f-bd9a-48b6-a59d-944680e2d6a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:19.601226 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf64142f-bd9a-48b6-a59d-944680e2d6a6 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:19.617410 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 307/608] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:33:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:19.617410 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2296f05-095a-4528-b8a7-08998f4e251e None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:19.617890 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2296f05-095a-4528-b8a7-08998f4e251e None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:19.618545 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2296f05-095a-4528-b8a7-08998f4e251e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:19.636606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-c2296f05-095a-4528-b8a7-08998f4e251e None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:19.637704 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c2296f05-095a-4528-b8a7-08998f4e251e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:19.691127 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492fcd1-80bf-4942-8235-7c10bc640dfb None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:19.707506 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:19.708144 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 308/609] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:33:19.709003 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2296f05-095a-4528-b8a7-08998f4e251e None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:19.735843 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274031f6-e2e2-4ffc-8f3d-c0deef9d8bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:19.737768 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274031f6-e2e2-4ffc-8f3d-c0deef9d8bd4 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:19.738284 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274031f6-e2e2-4ffc-8f3d-c0deef9d8bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:19.738777 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274031f6-e2e2-4ffc-8f3d-c0deef9d8bd4 None admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:19.739855 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274031f6-e2e2-4ffc-8f3d-c0deef9d8bd4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:19.740381 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274031f6-e2e2-4ffc-8f3d-c0deef9d8bd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:19.742145 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274031f6-e2e2-4ffc-8f3d-c0deef9d8bd4 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:19.769738 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 302/610] 10.222.0.17 () {56 vars in 1149 bytes} [Mon Oct 17 03:33:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:19.775604 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274031f6-e2e2-4ffc-8f3d-c0deef9d8bd4 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:19.844966 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:19.852876 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2296f05-095a-4528-b8a7-08998f4e251e None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:19.883558 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274031f6-e2e2-4ffc-8f3d-c0deef9d8bd4 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:19.891232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4e7f7-de1b-4b31-9b65-54fb267aabda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:19.892784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4e7f7-de1b-4b31-9b65-54fb267aabda None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:19.893295 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4e7f7-de1b-4b31-9b65-54fb267aabda None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:19.893767 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c4e7f7-de1b-4b31-9b65-54fb267aabda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:19.908562 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4e7f7-de1b-4b31-9b65-54fb267aabda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:19.910944 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c4e7f7-de1b-4b31-9b65-54fb267aabda None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:19.920479 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5cc7c-2383-47ca-821c-213a28f2372b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:19.921130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5cc7c-2383-47ca-821c-213a28f2372b None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:19.921411 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5cc7c-2383-47ca-821c-213a28f2372b None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:19.921678 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a5cc7c-2383-47ca-821c-213a28f2372b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:19.931683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-b4c4e7f7-de1b-4b31-9b65-54fb267aabda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:19.931683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:19.931683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:19.931683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:19.931683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:19.931683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:19.931683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:19.931683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:19.931683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:19.931683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:19.931683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:19.931683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:19.931683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:19.931683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:19.931683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:19.931683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:19.932607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:19.932607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:19.932607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:19.932607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:19.932607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:19.932607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:19.932607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:19.932607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:19.932607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:19.932607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:19.932607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:19.932607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:19.932607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:19.932607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:19.932607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:19.932607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:19.932607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:19.933562 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:19.933562 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:19.933562 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:19.936142 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 309/611] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:19.937171 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4e7f7-de1b-4b31-9b65-54fb267aabda None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:19.966915 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:20.001767 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5cc7c-2383-47ca-821c-213a28f2372b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:20.003203 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a5cc7c-2383-47ca-821c-213a28f2372b None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:20.062348 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c4e7f7-de1b-4b31-9b65-54fb267aabda None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:20.098409 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4b12b-0410-45de-b208-2f652a22bc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:20.099058 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4b12b-0410-45de-b208-2f652a22bc36 None admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:20.099339 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4b12b-0410-45de-b208-2f652a22bc36 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:20.099607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4b12b-0410-45de-b208-2f652a22bc36 None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:20.100286 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4b12b-0410-45de-b208-2f652a22bc36 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:20.102770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac4b12b-0410-45de-b208-2f652a22bc36 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:20.109068 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 303/612] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:33:19 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 03:33:20.109914 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5cc7c-2383-47ca-821c-213a28f2372b None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:20.161061 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 310/613] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Oct 17 03:33:19 2022] POST /identity/v3/services => generated 269 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:33:20.161895 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4b12b-0410-45de-b208-2f652a22bc36 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:20.164532 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:20.191812 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:20.239697 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a5cc7c-2383-47ca-821c-213a28f2372b None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:20.286856 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04012d-a607-4bad-b566-b98634dd5d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:20.287525 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04012d-a607-4bad-b566-b98634dd5d7b None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:20.287932 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04012d-a607-4bad-b566-b98634dd5d7b None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:20.288349 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04012d-a607-4bad-b566-b98634dd5d7b None admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:20.289102 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04012d-a607-4bad-b566-b98634dd5d7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:20.289500 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04012d-a607-4bad-b566-b98634dd5d7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:20.291531 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04012d-a607-4bad-b566-b98634dd5d7b None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:20.307833 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4b12b-0410-45de-b208-2f652a22bc36 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:20.309812 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 304/614] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:20.310737 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04012d-a607-4bad-b566-b98634dd5d7b None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:20.347766 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d68e3-7324-4c90-acd1-69767b313051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:20.348874 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d68e3-7324-4c90-acd1-69767b313051 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:20.349149 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d68e3-7324-4c90-acd1-69767b313051 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:20.349427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9d68e3-7324-4c90-acd1-69767b313051 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:20.350022 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d68e3-7324-4c90-acd1-69767b313051 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:20.351372 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9d68e3-7324-4c90-acd1-69767b313051 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:20.361742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-3b9d68e3-7324-4c90-acd1-69767b313051 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 03:33:20.361742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:20.361742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:33:20.361742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:33:20.361742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:33:20.361742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:33:20.361742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:33:20.361742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:20.361742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 03:33:20.361742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:20.361742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:33:20.361742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:20.361742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:20.361742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:20.361742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:20.361742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:20.361742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:20.362732 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:20.362732 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:20.362732 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:20.362732 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:20.362732 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:20.362732 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:20.362732 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:33:20.362732 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:33:20.362732 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:33:20.362732 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:33:20.362732 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:20.362732 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:20.362732 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:20.362732 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:20.362732 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:20.362732 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:20.363544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:20.363544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:20.363544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:20.363544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:20.363544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:20.363544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:20.363544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:20.363544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:20.363544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:20.363544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:20.363544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:33:20.363544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:20.363544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 17 03:33:20.363544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:20.363544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 311/615] 10.222.0.17 () {56 vars in 1129 bytes} [Mon Oct 17 03:33:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:20.365125 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d68e3-7324-4c90-acd1-69767b313051 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:20.440336 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:20.445517 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04012d-a607-4bad-b566-b98634dd5d7b None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:20.473088 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:20.483635 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9d68e3-7324-4c90-acd1-69767b313051 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:20.484706 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a0861-2523-42e9-94c9-4fb771416d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:20.485644 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a0861-2523-42e9-94c9-4fb771416d38 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:20.486198 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a0861-2523-42e9-94c9-4fb771416d38 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:20.486672 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1a0861-2523-42e9-94c9-4fb771416d38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:20.497431 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1a0861-2523-42e9-94c9-4fb771416d38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:20.499377 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1a0861-2523-42e9-94c9-4fb771416d38 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:20.517209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-6c1a0861-2523-42e9-94c9-4fb771416d38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:20.517209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:20.517209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:20.517209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:20.517209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:20.517209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:20.517209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:20.517209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:20.517209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:20.517209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:20.517209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:20.517209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:20.517209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:20.517209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:20.517209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:20.517209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:20.518167 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:20.518167 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:20.518167 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:20.518167 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:20.518167 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:20.518167 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:20.518167 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:20.518167 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:20.518167 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:20.518167 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:20.518167 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:20.518167 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:20.518167 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:20.518167 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:20.518167 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:20.518167 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:20.518167 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:20.519025 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:20.519025 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:20.519025 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:20.525683 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 305/616] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:20.525683 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a0861-2523-42e9-94c9-4fb771416d38 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:20.527727 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c4ad0-e604-42a3-82ce-224e5ece0e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:20.528525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c4ad0-e604-42a3-82ce-224e5ece0e78 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:20.529012 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c4ad0-e604-42a3-82ce-224e5ece0e78 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:20.529380 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c4ad0-e604-42a3-82ce-224e5ece0e78 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:20.537542 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c4ad0-e604-42a3-82ce-224e5ece0e78 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:20.539387 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c4ad0-e604-42a3-82ce-224e5ece0e78 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:20.553793 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-630c4ad0-e604-42a3-82ce-224e5ece0e78 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 17 03:33:20.553793 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:20.553793 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:20.553793 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:20.553793 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:20.553793 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:20.553793 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:20.553793 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:20.553793 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:20.553793 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:20.553793 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:20.553793 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:20.553793 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:33:20.553793 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:33:20.553793 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:33:20.553793 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:33:20.554932 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:20.554932 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:20.554932 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:33:20.554932 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:20.554932 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:33:20.554932 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:20.554932 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:20.554932 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:20.554932 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:20.554932 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:20.554932 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:20.554932 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:20.554932 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:20.554932 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:20.554932 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:20.554932 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:20.554932 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:20.554932 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:20.556160 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:20.556160 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:20.556160 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:33:20.556160 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:33:20.556160 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:33:20.556160 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:33:20.556160 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:33:20.556160 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:33:20.556160 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 17 03:33:20.556160 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:20.558352 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 312/617] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Oct 17 03:33:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:20.559407 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c4ad0-e604-42a3-82ce-224e5ece0e78 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:20.638061 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1a0861-2523-42e9-94c9-4fb771416d38 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:20.675535 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a170205-4b4a-4b1a-93c1-cc9c2d23969e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:20.676415 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a170205-4b4a-4b1a-93c1-cc9c2d23969e None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:20.676882 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a170205-4b4a-4b1a-93c1-cc9c2d23969e None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:20.677306 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a170205-4b4a-4b1a-93c1-cc9c2d23969e None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:20.677781 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c4ad0-e604-42a3-82ce-224e5ece0e78 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:20.678247 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a170205-4b4a-4b1a-93c1-cc9c2d23969e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:20.678757 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a170205-4b4a-4b1a-93c1-cc9c2d23969e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:20.681106 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a170205-4b4a-4b1a-93c1-cc9c2d23969e None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:20.686681 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 306/618] 10.222.0.17 () {56 vars in 1124 bytes} [Mon Oct 17 03:33:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:20.687815 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a170205-4b4a-4b1a-93c1-cc9c2d23969e None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:20.713249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:20.722976 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a6f34-7536-442b-8c54-402342da8daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:20.725264 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a6f34-7536-442b-8c54-402342da8daa None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:20.725264 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a6f34-7536-442b-8c54-402342da8daa None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:20.725264 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a6f34-7536-442b-8c54-402342da8daa None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:20.725264 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a6f34-7536-442b-8c54-402342da8daa None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:20.725696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a6f34-7536-442b-8c54-402342da8daa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:20.727777 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a6f34-7536-442b-8c54-402342da8daa None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:20.743220 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 313/619] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:20.744126 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a6f34-7536-442b-8c54-402342da8daa None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:20.746949 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:20.819945 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a170205-4b4a-4b1a-93c1-cc9c2d23969e None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:20.861499 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a6f34-7536-442b-8c54-402342da8daa None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:20.866235 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15915ef-6338-4ed5-a9d4-b4774bf038b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:20.866944 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15915ef-6338-4ed5-a9d4-b4774bf038b4 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:20.867228 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15915ef-6338-4ed5-a9d4-b4774bf038b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:20.867496 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15915ef-6338-4ed5-a9d4-b4774bf038b4 None admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:20.868806 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15915ef-6338-4ed5-a9d4-b4774bf038b4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:20.869108 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15915ef-6338-4ed5-a9d4-b4774bf038b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:20.870531 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15915ef-6338-4ed5-a9d4-b4774bf038b4 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:20.891796 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 307/620] 10.222.0.17 () {56 vars in 1135 bytes} [Mon Oct 17 03:33:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:20.893057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15915ef-6338-4ed5-a9d4-b4774bf038b4 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:20.906953 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ba67c-d397-414e-b3aa-29adbe747fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:20.907632 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ba67c-d397-414e-b3aa-29adbe747fd5 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:20.907936 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ba67c-d397-414e-b3aa-29adbe747fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:20.908202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ba67c-d397-414e-b3aa-29adbe747fd5 None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:20.909085 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ba67c-d397-414e-b3aa-29adbe747fd5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:20.909405 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ba67c-d397-414e-b3aa-29adbe747fd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:20.911220 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ba67c-d397-414e-b3aa-29adbe747fd5 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:20.917117 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 314/621] 10.222.0.17 () {56 vars in 1124 bytes} [Mon Oct 17 03:33:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:20.917917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ba67c-d397-414e-b3aa-29adbe747fd5 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:20.982340 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:21.003173 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:21.007751 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15915ef-6338-4ed5-a9d4-b4774bf038b4 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:21.042771 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ba67c-d397-414e-b3aa-29adbe747fd5 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:21.046926 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd44690-3e6b-460d-8e87-e6af319b1ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:21.047768 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd44690-3e6b-460d-8e87-e6af319b1ad8 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:21.048228 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd44690-3e6b-460d-8e87-e6af319b1ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:21.048771 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd44690-3e6b-460d-8e87-e6af319b1ad8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:21.056999 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd44690-3e6b-460d-8e87-e6af319b1ad8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:21.058932 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd44690-3e6b-460d-8e87-e6af319b1ad8 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:21.071664 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-7bd44690-3e6b-460d-8e87-e6af319b1ad8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:21.071664 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:21.071664 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:21.071664 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:21.071664 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:21.071664 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:21.071664 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:21.071664 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:21.071664 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:21.071664 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:21.071664 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:21.071664 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:21.071664 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:33:21.071664 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:33:21.071664 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:33:21.072861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:33:21.072861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:21.072861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:21.072861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:21.072861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:21.072861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:21.072861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:21.072861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:21.072861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:21.072861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:21.072861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:21.072861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:21.072861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:21.072861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:21.072861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:21.072861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:21.072861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:21.073937 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:33:21.073937 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:33:21.073937 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:33:21.073937 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:33:21.073937 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:33:21.073937 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:33:21.073937 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:21.073937 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:21.075893 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 308/622] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:33:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:21.076907 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd44690-3e6b-460d-8e87-e6af319b1ad8 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:21.078523 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40701570-276b-4d87-84c3-a567ec7465af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:21.079132 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40701570-276b-4d87-84c3-a567ec7465af None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:21.079584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40701570-276b-4d87-84c3-a567ec7465af None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:21.079895 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40701570-276b-4d87-84c3-a567ec7465af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:21.155074 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40701570-276b-4d87-84c3-a567ec7465af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:21.156581 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40701570-276b-4d87-84c3-a567ec7465af None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:21.184591 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd44690-3e6b-460d-8e87-e6af319b1ad8 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:21.209142 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:21.221968 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc18f6f-b1b2-47d7-a2c3-405b37840db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:21.222693 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc18f6f-b1b2-47d7-a2c3-405b37840db1 None admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:21.234428 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc18f6f-b1b2-47d7-a2c3-405b37840db1 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:21.234428 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc18f6f-b1b2-47d7-a2c3-405b37840db1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:21.234428 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc18f6f-b1b2-47d7-a2c3-405b37840db1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:21.234428 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc18f6f-b1b2-47d7-a2c3-405b37840db1 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:21.260993 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 315/623] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:33:20 2022] GET /identity/v3/auth/tokens => generated 1112 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 03:33:21.262373 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12beb045-09bf-4b97-8971-591a8124e72c None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:21.262660 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12beb045-09bf-4b97-8971-591a8124e72c None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:21.262929 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12beb045-09bf-4b97-8971-591a8124e72c None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:21.264329 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 316/624] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:21.264937 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12beb045-09bf-4b97-8971-591a8124e72c None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:21.327775 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 309/625] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:33:21 2022] POST /identity/v3/endpoints => generated 346 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:33:21.327775 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc18f6f-b1b2-47d7-a2c3-405b37840db1 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:21.345278 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:21.403598 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12beb045-09bf-4b97-8971-591a8124e72c None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:21.455394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb3c93-3376-4e58-8eae-1377bcf137a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:21.455883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc18f6f-b1b2-47d7-a2c3-405b37840db1 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:21.456351 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb3c93-3376-4e58-8eae-1377bcf137a9 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:21.456351 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb3c93-3376-4e58-8eae-1377bcf137a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:21.456623 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb3c93-3376-4e58-8eae-1377bcf137a9 None admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:21.457344 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb3c93-3376-4e58-8eae-1377bcf137a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:21.457655 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb3c93-3376-4e58-8eae-1377bcf137a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:21.460496 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb3c93-3376-4e58-8eae-1377bcf137a9 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:21.478113 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 317/626] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Oct 17 03:33:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:21.480074 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3864f-b141-4d21-af62-e92934dd7513 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:21.480366 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3864f-b141-4d21-af62-e92934dd7513 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:21.480644 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3864f-b141-4d21-af62-e92934dd7513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:21.499728 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:21.502628 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2e458-3713-488f-abe3-24e2760bf024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:21.503575 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2e458-3713-488f-abe3-24e2760bf024 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:21.504085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2e458-3713-488f-abe3-24e2760bf024 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:21.504698 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2e458-3713-488f-abe3-24e2760bf024 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:21.508311 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-b3f3864f-b141-4d21-af62-e92934dd7513 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:21.508311 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-b3f3864f-b141-4d21-af62-e92934dd7513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:21.516980 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2e458-3713-488f-abe3-24e2760bf024 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:21.518799 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2e458-3713-488f-abe3-24e2760bf024 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:21.536623 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-d1e2e458-3713-488f-abe3-24e2760bf024 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:21.536623 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:21.536623 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:21.536623 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:21.536623 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:21.536623 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:21.536623 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:21.536623 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:21.536623 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:21.536623 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:21.536623 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:21.536623 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:21.536623 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:21.536623 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:21.536623 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:21.536623 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:21.537600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:21.537600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:21.537600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:21.537600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:21.537600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:21.537600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:21.537600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:21.537600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:21.537600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:21.537600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:21.537600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:21.537600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:21.537600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:21.537600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:21.537600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:21.537600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:21.537600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:21.538495 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:21.538495 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:21.538495 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:21.538495 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 310/627] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:21.538495 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2e458-3713-488f-abe3-24e2760bf024 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:21.595808 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 318/628] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:21 2022] POST /identity/v3/auth/tokens => generated 1112 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:33:21.596678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3864f-b141-4d21-af62-e92934dd7513 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:21.639557 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:21.688576 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2e458-3713-488f-abe3-24e2760bf024 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:21.708038 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3864f-b141-4d21-af62-e92934dd7513 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:21.725748 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1171e-e952-4bb1-b58e-86e49af17ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:21.726447 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1171e-e952-4bb1-b58e-86e49af17ec6 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:21.726775 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1171e-e952-4bb1-b58e-86e49af17ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:21.727615 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1171e-e952-4bb1-b58e-86e49af17ec6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:21.728447 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a1171e-e952-4bb1-b58e-86e49af17ec6 None admin] RBAC: Adding query filter params (role.id=8a0215180fe04d5a8f2c24b370382456, scope.project.id=932cdb9ef7644bf5ba57c6d1d6bce7ff, user.id=cecbc19355ef4570b2b05bba876a3a8b) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:21.728755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a1171e-e952-4bb1-b58e-86e49af17ec6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:21.730214 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a1171e-e952-4bb1-b58e-86e49af17ec6 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:21.736917 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 311/629] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Oct 17 03:33:21 2022] GET /identity/v3/role_assignments?user.id=cecbc19355ef4570b2b05bba876a3a8b&scope.project.id=932cdb9ef7644bf5ba57c6d1d6bce7ff&role.id=8a0215180fe04d5a8f2c24b370382456 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:21.750908 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9c902-8776-466d-b8c8-074924e690ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:21.751491 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9c902-8776-466d-b8c8-074924e690ea None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:21.751802 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9c902-8776-466d-b8c8-074924e690ea None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:21.752074 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9c902-8776-466d-b8c8-074924e690ea None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:21.752724 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9c902-8776-466d-b8c8-074924e690ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:21.753017 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9c902-8776-466d-b8c8-074924e690ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:21.754381 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9c902-8776-466d-b8c8-074924e690ea None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:21.757236 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:21.768012 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 319/630] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:21.776957 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1171e-e952-4bb1-b58e-86e49af17ec6 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:21.918268 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1171e-e952-4bb1-b58e-86e49af17ec6 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:21.937838 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9c902-8776-466d-b8c8-074924e690ea None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:21.950997 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:21.967385 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac28b8e-ed1b-4e68-9cba-929ee53de990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:21.968516 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac28b8e-ed1b-4e68-9cba-929ee53de990 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:21.969198 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac28b8e-ed1b-4e68-9cba-929ee53de990 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:21.970985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac28b8e-ed1b-4e68-9cba-929ee53de990 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:21.982289 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac28b8e-ed1b-4e68-9cba-929ee53de990 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:21.984259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac28b8e-ed1b-4e68-9cba-929ee53de990 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:22.001538 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-6ac28b8e-ed1b-4e68-9cba-929ee53de990 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:22.001538 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:22.001538 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:22.001538 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:22.001538 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:22.001538 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:22.001538 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:22.001538 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:22.001538 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:22.001538 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:22.001538 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:22.001538 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:22.001538 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:33:22.001538 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:33:22.001538 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:33:22.002422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:33:22.002422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:22.002422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:22.002422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:22.002422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:22.002422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:22.002422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:22.002422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:22.002422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:22.002422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:22.002422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:22.002422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:22.002422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:22.002422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:22.002422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:22.002422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:22.002422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:22.003290 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:33:22.003290 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:33:22.003290 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:33:22.003290 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:33:22.003290 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:33:22.003290 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:33:22.003290 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:22.003290 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:22.005762 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 312/631] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:33:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:22.012023 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac28b8e-ed1b-4e68-9cba-929ee53de990 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:22.066734 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9c902-8776-466d-b8c8-074924e690ea None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:22.106270 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12fe88a-4538-483b-b578-e459ee9dd5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:22.106964 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fe88a-4538-483b-b578-e459ee9dd5bf None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:22.107240 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fe88a-4538-483b-b578-e459ee9dd5bf None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:22.107556 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fe88a-4538-483b-b578-e459ee9dd5bf None admin] PATH_INFO: `/v3/services/network` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:22.108169 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12fe88a-4538-483b-b578-e459ee9dd5bf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:22.109516 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12fe88a-4538-483b-b578-e459ee9dd5bf None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:22.119278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-a12fe88a-4538-483b-b578-e459ee9dd5bf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 03:33:22.119278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:22.119278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:33:22.119278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:33:22.119278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:33:22.119278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:33:22.119278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:33:22.119278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:22.119278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 03:33:22.119278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:22.119278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:33:22.119278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:22.119278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:22.119278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:22.119278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:22.119278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:22.119278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:22.120303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:22.120303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:22.120303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:22.120303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:22.120303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:22.120303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:22.120303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:33:22.120303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:33:22.120303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:33:22.120303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:33:22.120303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:22.120303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:22.120303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:22.120303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:22.120303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:22.120303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:22.121116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:22.121116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:22.121116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:22.121116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:22.121116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:22.121116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:22.121116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:22.121116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:22.121116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:22.121116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:22.121116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:33:22.121116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:22.121116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 17 03:33:22.121116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:22.121824 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 320/632] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:33:21 2022] GET /identity/v3/services/network => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:33:22.127128 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12fe88a-4538-483b-b578-e459ee9dd5bf None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:22.151103 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac28b8e-ed1b-4e68-9cba-929ee53de990 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:22.195237 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98162924-eb72-4e08-a27f-0e2e97680847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:22.196791 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98162924-eb72-4e08-a27f-0e2e97680847 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:22.197284 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98162924-eb72-4e08-a27f-0e2e97680847 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:22.197759 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98162924-eb72-4e08-a27f-0e2e97680847 None admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:22.198939 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98162924-eb72-4e08-a27f-0e2e97680847 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:22.199423 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98162924-eb72-4e08-a27f-0e2e97680847 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:22.201316 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98162924-eb72-4e08-a27f-0e2e97680847 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:22.211725 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:22.218575 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 313/633] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Oct 17 03:33:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:33:22.225313 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98162924-eb72-4e08-a27f-0e2e97680847 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:22.284457 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:22.299274 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12fe88a-4538-483b-b578-e459ee9dd5bf None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:22.339402 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98162924-eb72-4e08-a27f-0e2e97680847 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:22.375206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd43f3-a313-4987-ab58-2b2b76e62e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:22.376179 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd43f3-a313-4987-ab58-2b2b76e62e5d None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:22.376427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8edd8-6376-4906-bac3-721ac1dd76c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:22.376865 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd43f3-a313-4987-ab58-2b2b76e62e5d None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:22.377047 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8edd8-6376-4906-bac3-721ac1dd76c7 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:22.377327 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8edd8-6376-4906-bac3-721ac1dd76c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:22.377597 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8edd8-6376-4906-bac3-721ac1dd76c7 None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:22.377861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd43f3-a313-4987-ab58-2b2b76e62e5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:22.378232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e8edd8-6376-4906-bac3-721ac1dd76c7 None admin] RBAC: Adding query filter params (name=network) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:22.378530 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e8edd8-6376-4906-bac3-721ac1dd76c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:22.379119 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd43f3-a313-4987-ab58-2b2b76e62e5d None admin] RBAC: Adding query filter params (role.id=611120afb3a544d0b5161da1b769a2f8, scope.project.id=932cdb9ef7644bf5ba57c6d1d6bce7ff, user.id=5f64aaf4d408462a88963973d42abfb9) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:22.379607 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd43f3-a313-4987-ab58-2b2b76e62e5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:22.379914 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e8edd8-6376-4906-bac3-721ac1dd76c7 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:22.381571 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fd43f3-a313-4987-ab58-2b2b76e62e5d None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:22.388682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 314/634] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Oct 17 03:33:22 2022] GET /identity/v3/role_assignments?user.id=5f64aaf4d408462a88963973d42abfb9&scope.project.id=932cdb9ef7644bf5ba57c6d1d6bce7ff&role.id=611120afb3a544d0b5161da1b769a2f8 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:33:22.390626 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd43f3-a313-4987-ab58-2b2b76e62e5d None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:22.392569 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 321/635] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Oct 17 03:33:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:33:22.523808 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd43f3-a313-4987-ab58-2b2b76e62e5d None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:22.552315 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:22.572356 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e524c8-a4c2-4afc-aa99-4b4aea631daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:22.573142 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e524c8-a4c2-4afc-aa99-4b4aea631daa None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:22.573427 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e524c8-a4c2-4afc-aa99-4b4aea631daa None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:22.573701 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e524c8-a4c2-4afc-aa99-4b4aea631daa None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:22.574362 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e524c8-a4c2-4afc-aa99-4b4aea631daa None admin] RBAC: Adding query filter params (type=network) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:22.574663 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e524c8-a4c2-4afc-aa99-4b4aea631daa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:22.576137 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e524c8-a4c2-4afc-aa99-4b4aea631daa None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:22.582216 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 315/636] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Oct 17 03:33:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:33:22.587878 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8edd8-6376-4906-bac3-721ac1dd76c7 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:22.629376 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96808e-3994-445e-a0b5-e8b954c99f8e None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:22.629787 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96808e-3994-445e-a0b5-e8b954c99f8e None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:22.630085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96808e-3994-445e-a0b5-e8b954c99f8e None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:22.631428 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 316/637] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:33:22.638284 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f09b5-cd3b-48e2-af73-4f0db6fd58a8 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:22.639395 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f09b5-cd3b-48e2-af73-4f0db6fd58a8 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:22.640027 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f09b5-cd3b-48e2-af73-4f0db6fd58a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:22.664544 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-921f09b5-cd3b-48e2-af73-4f0db6fd58a8 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:22.666073 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-921f09b5-cd3b-48e2-af73-4f0db6fd58a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:22.743984 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 317/638] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:22 2022] POST /identity/v3/auth/tokens => generated 1112 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:33:22.758047 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8edd8-6376-4906-bac3-721ac1dd76c7 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:22.805189 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:22.818087 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e9101b-2710-42d4-a439-ef41926a04bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:22.819386 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9101b-2710-42d4-a439-ef41926a04bb None admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:22.819386 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9101b-2710-42d4-a439-ef41926a04bb None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:22.819386 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9101b-2710-42d4-a439-ef41926a04bb None admin] PATH_INFO: `/v3/endpoints` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:22.820078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e9101b-2710-42d4-a439-ef41926a04bb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:22.821961 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e9101b-2710-42d4-a439-ef41926a04bb None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:22.905233 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 322/639] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:33:22 2022] POST /identity/v3/endpoints => generated 337 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:33:23.124031 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f09b5-cd3b-48e2-af73-4f0db6fd58a8 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:23.245370 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f09b5-cd3b-48e2-af73-4f0db6fd58a8 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:23.257989 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:23.297383 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567699ca-c0c9-44de-be3d-fe75c812fcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:23.298027 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567699ca-c0c9-44de-be3d-fe75c812fcfa None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:23.298305 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567699ca-c0c9-44de-be3d-fe75c812fcfa None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:23.298572 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567699ca-c0c9-44de-be3d-fe75c812fcfa None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:23.299157 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567699ca-c0c9-44de-be3d-fe75c812fcfa None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:23.300594 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567699ca-c0c9-44de-be3d-fe75c812fcfa None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:23.313960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-567699ca-c0c9-44de-be3d-fe75c812fcfa None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 03:33:23.313960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:23.313960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:33:23.313960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:33:23.313960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:33:23.313960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:33:23.313960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:33:23.313960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:23.313960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 03:33:23.313960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:23.313960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:33:23.313960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:23.313960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:23.313960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:23.313960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:23.313960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:23.314969 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:23.314969 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:23.314969 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:23.314969 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:23.314969 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:23.314969 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:23.314969 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:23.314969 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:33:23.314969 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:33:23.314969 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:33:23.314969 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:33:23.314969 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:23.314969 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:23.314969 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:23.314969 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:23.314969 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:23.314969 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:23.315945 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:23.315945 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:23.315945 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:23.315945 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:23.315945 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:23.315945 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:23.315945 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:23.315945 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:23.315945 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:23.315945 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:23.315945 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:33:23.315945 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:23.315945 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 03:33:23.315945 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:23.315945 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 318/640] 10.222.0.17 () {56 vars in 1132 bytes} [Mon Oct 17 03:33:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:23.321440 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e9101b-2710-42d4-a439-ef41926a04bb None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:23.441144 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e9101b-2710-42d4-a439-ef41926a04bb None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:23.476666 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16553bde-bb91-4b5d-9223-c14716d29ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:23.477320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16553bde-bb91-4b5d-9223-c14716d29ada None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:23.477605 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16553bde-bb91-4b5d-9223-c14716d29ada None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:23.477878 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16553bde-bb91-4b5d-9223-c14716d29ada None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:23.478518 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16553bde-bb91-4b5d-9223-c14716d29ada None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:23.478822 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16553bde-bb91-4b5d-9223-c14716d29ada None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:23.480231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16553bde-bb91-4b5d-9223-c14716d29ada None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:23.485914 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 323/641] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Oct 17 03:33:23 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:23.491419 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567699ca-c0c9-44de-be3d-fe75c812fcfa None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:23.602704 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567699ca-c0c9-44de-be3d-fe75c812fcfa None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:23.639336 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345dc99a-0c95-49b6-bd54-f09bae42947a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:23.640312 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345dc99a-0c95-49b6-bd54-f09bae42947a None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:23.640802 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345dc99a-0c95-49b6-bd54-f09bae42947a None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:23.641275 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345dc99a-0c95-49b6-bd54-f09bae42947a None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:23.642123 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345dc99a-0c95-49b6-bd54-f09bae42947a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:23.642617 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345dc99a-0c95-49b6-bd54-f09bae42947a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:23.644184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345dc99a-0c95-49b6-bd54-f09bae42947a None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:23.649767 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 319/642] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Oct 17 03:33:23 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:23.657950 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16553bde-bb91-4b5d-9223-c14716d29ada None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:23.713805 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:23.770961 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16553bde-bb91-4b5d-9223-c14716d29ada None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:23.806071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2953e899-f528-4492-aec8-85498ea99b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:23.806682 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953e899-f528-4492-aec8-85498ea99b11 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:23.806963 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953e899-f528-4492-aec8-85498ea99b11 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:23.807253 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2953e899-f528-4492-aec8-85498ea99b11 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:23.807961 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2953e899-f528-4492-aec8-85498ea99b11 None admin] RBAC: Adding query filter params (interface=public, service_id=8f56c22c28754726a43ead24132b782f, region_id=RegionOne) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:23.808270 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2953e899-f528-4492-aec8-85498ea99b11 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:23.809694 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2953e899-f528-4492-aec8-85498ea99b11 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:23.815153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 324/643] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Oct 17 03:33:23 2022] GET /identity/v3/endpoints?service_id=8f56c22c28754726a43ead24132b782f&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:23.820480 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345dc99a-0c95-49b6-bd54-f09bae42947a None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:23.858450 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:23.933987 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345dc99a-0c95-49b6-bd54-f09bae42947a None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:23.959796 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c4025-664f-4a7c-b6cc-8c24ec729ee3 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:23.960122 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c4025-664f-4a7c-b6cc-8c24ec729ee3 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:23.960348 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c4025-664f-4a7c-b6cc-8c24ec729ee3 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:23.961965 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 325/644] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:23.967211 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e948a9-f46a-4f9b-b58e-720bbe7da286 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:23.967437 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e948a9-f46a-4f9b-b58e-720bbe7da286 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:23.967712 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e948a9-f46a-4f9b-b58e-720bbe7da286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:23.970252 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d8e34-e012-4c8f-ac07-60d7f4712821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:23.970975 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d8e34-e012-4c8f-ac07-60d7f4712821 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:23.971254 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d8e34-e012-4c8f-ac07-60d7f4712821 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:23.971526 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d8e34-e012-4c8f-ac07-60d7f4712821 None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:23.972354 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d8e34-e012-4c8f-ac07-60d7f4712821 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:23.973764 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d8e34-e012-4c8f-ac07-60d7f4712821 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:23.980143 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 320/645] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Oct 17 03:33:23 2022] GET /identity/v3/services => generated 1029 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:33:23.987009 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-b3e948a9-f46a-4f9b-b58e-720bbe7da286 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:23.988281 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-b3e948a9-f46a-4f9b-b58e-720bbe7da286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:24.070639 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 326/646] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:23 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:33:24.341850 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d8e34-e012-4c8f-ac07-60d7f4712821 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:24.449010 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:24.455206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d8e34-e012-4c8f-ac07-60d7f4712821 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:24.492676 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08c073-455e-4e0b-aaec-fe2d6e108132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:24.493374 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08c073-455e-4e0b-aaec-fe2d6e108132 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:24.493656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08c073-455e-4e0b-aaec-fe2d6e108132 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:24.493938 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08c073-455e-4e0b-aaec-fe2d6e108132 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:24.494526 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08c073-455e-4e0b-aaec-fe2d6e108132 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:24.495907 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08c073-455e-4e0b-aaec-fe2d6e108132 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:24.504940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-bd08c073-455e-4e0b-aaec-fe2d6e108132 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 03:33:24.504940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:24.504940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:33:24.504940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:33:24.504940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:33:24.504940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:33:24.504940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:33:24.504940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:24.504940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 03:33:24.504940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:24.504940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:33:24.504940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:24.504940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:24.504940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:24.504940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:24.504940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:24.504940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:24.506857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:24.506857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:24.506857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:24.506857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:24.506857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:24.506857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:24.506857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:33:24.506857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:33:24.506857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:33:24.506857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:33:24.506857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:24.506857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:24.506857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:24.506857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:24.506857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:24.506857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:24.508015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:24.508015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:24.508015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:24.508015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:24.508015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:24.508015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:24.508015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:24.508015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:24.508015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:24.508015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:24.508015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:33:24.508015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:24.508015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 03:33:24.508015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:24.508015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 321/647] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Oct 17 03:33:24 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:24.513376 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e948a9-f46a-4f9b-b58e-720bbe7da286 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:24.536109 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237d330-b38f-443a-90b2-07e4b30704a3 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:24.536394 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237d330-b38f-443a-90b2-07e4b30704a3 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:24.536674 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237d330-b38f-443a-90b2-07e4b30704a3 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:24.538261 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 322/648] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:24.544687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39239ca8-732c-4622-a66e-3812aec8e51e None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:24.544687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39239ca8-732c-4622-a66e-3812aec8e51e None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:24.545140 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39239ca8-732c-4622-a66e-3812aec8e51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:24.557887 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:24.569070 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-39239ca8-732c-4622-a66e-3812aec8e51e None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:24.570332 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-39239ca8-732c-4622-a66e-3812aec8e51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:24.632043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e948a9-f46a-4f9b-b58e-720bbe7da286 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:24.650100 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 323/649] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:24 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:33:24.670784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f74a87-65a7-4fc2-9fa3-6c25c05c0338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:24.671448 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f74a87-65a7-4fc2-9fa3-6c25c05c0338 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:24.671730 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f74a87-65a7-4fc2-9fa3-6c25c05c0338 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:24.672078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f74a87-65a7-4fc2-9fa3-6c25c05c0338 None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:24.672718 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f74a87-65a7-4fc2-9fa3-6c25c05c0338 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:24.673023 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f74a87-65a7-4fc2-9fa3-6c25c05c0338 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:24.674491 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f74a87-65a7-4fc2-9fa3-6c25c05c0338 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:24.685515 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 327/650] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Oct 17 03:33:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:24.693322 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39239ca8-732c-4622-a66e-3812aec8e51e None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:24.796170 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:24.808343 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39239ca8-732c-4622-a66e-3812aec8e51e None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:24.844736 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981285ae-2f2f-44ec-8e36-c26ec936cbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:24.845414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981285ae-2f2f-44ec-8e36-c26ec936cbfb None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:24.845700 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981285ae-2f2f-44ec-8e36-c26ec936cbfb None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:24.845972 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981285ae-2f2f-44ec-8e36-c26ec936cbfb None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:24.846948 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981285ae-2f2f-44ec-8e36-c26ec936cbfb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:24.847297 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981285ae-2f2f-44ec-8e36-c26ec936cbfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:24.848687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981285ae-2f2f-44ec-8e36-c26ec936cbfb None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:24.853784 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 324/651] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Oct 17 03:33:24 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:24.875911 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118e06d-6188-46cc-9910-52a06b5b4d66 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:24.876230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118e06d-6188-46cc-9910-52a06b5b4d66 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:24.876478 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3118e06d-6188-46cc-9910-52a06b5b4d66 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:24.877877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 328/652] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:24.886633 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a309d8-ba80-448e-addc-33e391fb35bd None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:24.886817 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a309d8-ba80-448e-addc-33e391fb35bd None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:24.887096 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a309d8-ba80-448e-addc-33e391fb35bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:24.910043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-36a309d8-ba80-448e-addc-33e391fb35bd None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:24.911625 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-36a309d8-ba80-448e-addc-33e391fb35bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:24.927601 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3118e06d-6188-46cc-9910-52a06b5b4d66 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:25.003420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 325/653] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:24 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:33:25.046419 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3118e06d-6188-46cc-9910-52a06b5b4d66 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:25.048267 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:25.085242 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4af3b-d4e9-4004-b93a-bff4be72b35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:25.085879 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4af3b-d4e9-4004-b93a-bff4be72b35d None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:25.086769 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4af3b-d4e9-4004-b93a-bff4be72b35d None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:25.087046 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c4af3b-d4e9-4004-b93a-bff4be72b35d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:25.098014 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c4af3b-d4e9-4004-b93a-bff4be72b35d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:33:25.103045 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-a5c4af3b-d4e9-4004-b93a-bff4be72b35d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 03:33:25.103045 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:25.103045 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:25.103045 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:25.103045 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:25.103045 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:25.103045 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:25.103045 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:25.103045 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:25.103045 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:25.103045 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:25.103045 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:25.103045 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:33:25.103045 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:33:25.103045 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:33:25.103045 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:33:25.104007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:33:25.104007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:33:25.104007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:33:25.104007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:33:25.104007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:25.104007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:25.104007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:25.104007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:25.104007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:25.104007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:25.104007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:25.104007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:25.104007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:25.104007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:25.104007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:25.104007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:25.104817 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:25.104817 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:25.104817 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:25.104817 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:25.104817 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:33:25.104817 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:33:25.104817 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:33:25.104817 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:33:25.104817 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:33:25.104817 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:33:25.104817 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 03:33:25.104817 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:25.104817 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 329/654] 10.222.0.17 () {56 vars in 1120 bytes} [Mon Oct 17 03:33:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:33:25.112131 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a309d8-ba80-448e-addc-33e391fb35bd None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:25.199638 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:25.220110 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a309d8-ba80-448e-addc-33e391fb35bd None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:25.257540 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b5162-f83f-4f72-b92c-d722818ca420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:25.258701 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b5162-f83f-4f72-b92c-d722818ca420 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:25.258986 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b5162-f83f-4f72-b92c-d722818ca420 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:25.259263 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b5162-f83f-4f72-b92c-d722818ca420 None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:25.259959 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b5162-f83f-4f72-b92c-d722818ca420 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:25.260448 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b5162-f83f-4f72-b92c-d722818ca420 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:25.261818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b5162-f83f-4f72-b92c-d722818ca420 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:25.270788 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 326/655] 10.222.0.17 () {56 vars in 1115 bytes} [Mon Oct 17 03:33:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:25.276142 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4af3b-d4e9-4004-b93a-bff4be72b35d None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:25.289545 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b5162-f83f-4f72-b92c-d722818ca420 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:25.382122 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c4af3b-d4e9-4004-b93a-bff4be72b35d None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:25.399833 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b5162-f83f-4f72-b92c-d722818ca420 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:25.422297 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae349c-ebba-4b67-baed-88b2614badb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:25.423470 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae349c-ebba-4b67-baed-88b2614badb9 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:25.424222 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae349c-ebba-4b67-baed-88b2614badb9 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:25.425122 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dae349c-ebba-4b67-baed-88b2614badb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:25.436698 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae80ff5-a180-46f3-8a9b-2a251b43d65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:25.437333 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae80ff5-a180-46f3-8a9b-2a251b43d65a None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:25.437618 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae80ff5-a180-46f3-8a9b-2a251b43d65a None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:25.437893 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae80ff5-a180-46f3-8a9b-2a251b43d65a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:25.438473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae80ff5-a180-46f3-8a9b-2a251b43d65a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:25.439475 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dae349c-ebba-4b67-baed-88b2614badb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:25.439878 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae80ff5-a180-46f3-8a9b-2a251b43d65a None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:25.441676 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dae349c-ebba-4b67-baed-88b2614badb9 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:25.450057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-3ae80ff5-a180-46f3-8a9b-2a251b43d65a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:33:25.450057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:25.450057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:33:25.450057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:33:25.450057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:33:25.450057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:33:25.450057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:33:25.450057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:25.450057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:33:25.450057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:25.450057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:33:25.450057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:25.450057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:25.450057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:25.450057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:25.450057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:25.450057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:25.451097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:25.451097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:25.451097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:25.451097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:25.451097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:25.451097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:25.451097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:33:25.451097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:33:25.451097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:33:25.451097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:33:25.451097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:25.451097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:25.451097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:25.451097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:25.451097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:25.451097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:25.451961 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:25.451961 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:25.451961 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:25.451961 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:25.451961 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:25.451961 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:25.451961 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:25.451961 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:25.451961 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:25.451961 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:25.451961 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:33:25.451961 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:25.451961 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:33:25.451961 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:25.451961 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 327/656] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Oct 17 03:33:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:33:25.456765 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae80ff5-a180-46f3-8a9b-2a251b43d65a None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:25.461524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-9dae349c-ebba-4b67-baed-88b2614badb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:25.461524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:25.461524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:25.461524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:25.461524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:25.461524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:25.461524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:25.461524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:25.461524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:25.461524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:25.461524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:25.461524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:25.461524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:25.461524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:25.461524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:25.461524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:25.462455 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:25.462455 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:25.462455 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:25.462455 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:25.462455 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:25.462455 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:25.462455 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:25.462455 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:25.462455 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:25.462455 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:25.462455 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:25.462455 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:25.462455 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:25.462455 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:25.462455 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:25.462455 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:25.462455 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:25.463320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:25.463320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:25.463320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:25.466952 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 330/657] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:25.468135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae349c-ebba-4b67-baed-88b2614badb9 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:25.473207 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:25.485202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:25.568470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae80ff5-a180-46f3-8a9b-2a251b43d65a None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:25.572570 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dae349c-ebba-4b67-baed-88b2614badb9 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:25.605907 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026aeda7-fbc2-472f-81d5-c922709d82c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:25.606945 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026aeda7-fbc2-472f-81d5-c922709d82c4 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:25.607435 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026aeda7-fbc2-472f-81d5-c922709d82c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:25.607982 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026aeda7-fbc2-472f-81d5-c922709d82c4 None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:25.608830 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026aeda7-fbc2-472f-81d5-c922709d82c4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:25.609203 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c0003-5c58-4791-97db-a64c2d652773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:25.609600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026aeda7-fbc2-472f-81d5-c922709d82c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:25.609813 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c0003-5c58-4791-97db-a64c2d652773 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:25.610093 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c0003-5c58-4791-97db-a64c2d652773 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:25.610365 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c0003-5c58-4791-97db-a64c2d652773 None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:25.611008 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c0003-5c58-4791-97db-a64c2d652773 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:25.611311 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c0003-5c58-4791-97db-a64c2d652773 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:25.612168 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026aeda7-fbc2-472f-81d5-c922709d82c4 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:25.612679 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c0003-5c58-4791-97db-a64c2d652773 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:25.618697 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 328/658] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Oct 17 03:33:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:33:25.626395 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026aeda7-fbc2-472f-81d5-c922709d82c4 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:25.630145 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 331/659] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:33:25.631581 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c0003-5c58-4791-97db-a64c2d652773 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:25.712676 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:25.715918 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:25.739258 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c0003-5c58-4791-97db-a64c2d652773 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:25.742118 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026aeda7-fbc2-472f-81d5-c922709d82c4 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:25.776649 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0bf552-d560-46af-ae7b-ec9ecd7c4dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:25.777305 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0bf552-d560-46af-ae7b-ec9ecd7c4dd6 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:25.777588 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0bf552-d560-46af-ae7b-ec9ecd7c4dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:25.777869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0bf552-d560-46af-ae7b-ec9ecd7c4dd6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:25.778771 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee47a87-52c9-424d-95d5-df83c0275cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:25.779606 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee47a87-52c9-424d-95d5-df83c0275cd9 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:25.780065 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee47a87-52c9-424d-95d5-df83c0275cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:25.780515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee47a87-52c9-424d-95d5-df83c0275cd9 None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:25.781321 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee47a87-52c9-424d-95d5-df83c0275cd9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:25.781830 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee47a87-52c9-424d-95d5-df83c0275cd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:25.783399 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee47a87-52c9-424d-95d5-df83c0275cd9 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:25.786779 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0bf552-d560-46af-ae7b-ec9ecd7c4dd6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:25.788295 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0bf552-d560-46af-ae7b-ec9ecd7c4dd6 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:25.796267 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 329/660] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Oct 17 03:33:25 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:25.801886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-be0bf552-d560-46af-ae7b-ec9ecd7c4dd6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 03:33:25.801886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:25.801886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:25.801886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:25.801886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:25.801886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:25.801886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:25.801886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:25.801886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:25.801886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:25.801886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:25.801886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:25.801886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:33:25.801886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:33:25.801886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:33:25.801886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:33:25.802917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:25.802917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:25.802917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:33:25.802917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:25.802917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:33:25.802917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:25.802917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:25.802917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:25.802917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:25.802917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:25.802917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:25.802917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:25.802917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:25.802917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:25.802917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:25.802917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:25.802917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:25.802917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:25.803913 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:25.803913 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:25.803913 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:33:25.803913 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:33:25.803913 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:33:25.803913 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:33:25.803913 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:33:25.803913 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:33:25.803913 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 03:33:25.803913 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:25.807885 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 332/661] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Oct 17 03:33:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:33:25.810151 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee47a87-52c9-424d-95d5-df83c0275cd9 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:25.924673 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee47a87-52c9-424d-95d5-df83c0275cd9 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:25.962042 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed1814-0749-4520-9443-9358dceef0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:25.962875 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed1814-0749-4520-9443-9358dceef0b8 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:25.963183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed1814-0749-4520-9443-9358dceef0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:25.963574 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed1814-0749-4520-9443-9358dceef0b8 None admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:25.964703 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed1814-0749-4520-9443-9358dceef0b8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:25.966471 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed1814-0749-4520-9443-9358dceef0b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:25.966607 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ed1814-0749-4520-9443-9358dceef0b8 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:25.975786 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:25.986818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 330/662] 10.222.0.17 () {56 vars in 1149 bytes} [Mon Oct 17 03:33:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:25.992528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0bf552-d560-46af-ae7b-ec9ecd7c4dd6 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:26.098340 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0bf552-d560-46af-ae7b-ec9ecd7c4dd6 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:26.133250 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5dfb4a-dc49-4d82-955e-595dc92db8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:26.133882 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5dfb4a-dc49-4d82-955e-595dc92db8be None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:26.134160 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5dfb4a-dc49-4d82-955e-595dc92db8be None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:26.134432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5dfb4a-dc49-4d82-955e-595dc92db8be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:26.136508 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:26.147458 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5dfb4a-dc49-4d82-955e-595dc92db8be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:26.149758 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5dfb4a-dc49-4d82-955e-595dc92db8be None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:26.166472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-2d5dfb4a-dc49-4d82-955e-595dc92db8be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:26.166472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:26.166472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:26.166472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:26.166472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:26.166472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:26.166472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:26.166472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:26.166472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:26.166472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:26.166472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:26.166472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:26.166472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:26.166472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:26.166472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:26.166472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:26.167673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:26.167673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:26.167673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:26.167673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:26.167673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:26.167673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:26.167673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:26.167673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:26.167673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:26.167673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:26.167673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:26.167673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:26.167673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:26.167673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:26.167673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:26.167673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:26.167673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:26.168802 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:26.168802 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:26.168802 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:26.169773 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 333/663] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:26.175561 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed1814-0749-4520-9443-9358dceef0b8 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:26.281511 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed1814-0749-4520-9443-9358dceef0b8 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:26.297567 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a5830-fd6c-424b-bedb-a606ae96a942 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:26.297980 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a5830-fd6c-424b-bedb-a606ae96a942 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:26.298350 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33a5830-fd6c-424b-bedb-a606ae96a942 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:26.300042 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 334/664] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:33:26.307762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2debd-d412-48ed-85a5-cb80620dd97c None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:26.307762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2debd-d412-48ed-85a5-cb80620dd97c None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:26.307762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2debd-d412-48ed-85a5-cb80620dd97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:26.325346 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c0aea-1194-4f80-99d0-5a84cbe4d2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:26.326199 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c0aea-1194-4f80-99d0-5a84cbe4d2c7 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:26.326625 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c0aea-1194-4f80-99d0-5a84cbe4d2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:26.327057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c0aea-1194-4f80-99d0-5a84cbe4d2c7 None admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:26.327818 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-71c2debd-d412-48ed-85a5-cb80620dd97c None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:26.328055 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c0aea-1194-4f80-99d0-5a84cbe4d2c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:26.328498 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c0aea-1194-4f80-99d0-5a84cbe4d2c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:26.329313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-71c2debd-d412-48ed-85a5-cb80620dd97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:26.330159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c0aea-1194-4f80-99d0-5a84cbe4d2c7 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:26.343524 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 331/665] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:26.349569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c0aea-1194-4f80-99d0-5a84cbe4d2c7 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:26.419782 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 335/666] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:26 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:33:26.440076 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:26.482579 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c0aea-1194-4f80-99d0-5a84cbe4d2c7 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:26.517286 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fba98c-3fc9-4f3b-af27-b0c89945cf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:26.517927 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fba98c-3fc9-4f3b-af27-b0c89945cf46 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:26.518206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fba98c-3fc9-4f3b-af27-b0c89945cf46 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:26.518482 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fba98c-3fc9-4f3b-af27-b0c89945cf46 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:26.526116 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fba98c-3fc9-4f3b-af27-b0c89945cf46 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:26.527637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fba98c-3fc9-4f3b-af27-b0c89945cf46 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:26.539227 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-69fba98c-3fc9-4f3b-af27-b0c89945cf46 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:26.539227 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:26.539227 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:26.539227 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:26.539227 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:26.539227 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:26.539227 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:26.539227 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:26.539227 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:26.539227 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:26.539227 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:26.539227 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:26.539227 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:33:26.539227 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:33:26.539227 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:33:26.540189 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:33:26.540189 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:26.540189 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:26.540189 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:26.540189 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:26.540189 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:26.540189 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:26.540189 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:26.540189 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:26.540189 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:26.540189 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:26.540189 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:26.540189 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:26.540189 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:26.540189 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:26.540189 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:26.540189 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:26.541555 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:33:26.541555 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:33:26.541555 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:33:26.541555 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:33:26.541555 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:33:26.541555 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:33:26.541555 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:26.541555 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:26.541555 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 332/667] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:33:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:26.548084 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2debd-d412-48ed-85a5-cb80620dd97c None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:26.656050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2debd-d412-48ed-85a5-cb80620dd97c None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:26.674465 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:26.693937 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39854c-95c5-42a7-ad5d-2efd56e88516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:26.694566 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39854c-95c5-42a7-ad5d-2efd56e88516 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:26.694850 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39854c-95c5-42a7-ad5d-2efd56e88516 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:26.695134 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39854c-95c5-42a7-ad5d-2efd56e88516 None admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:26.696473 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39854c-95c5-42a7-ad5d-2efd56e88516 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:26.696786 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39854c-95c5-42a7-ad5d-2efd56e88516 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:26.700361 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39854c-95c5-42a7-ad5d-2efd56e88516 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:26.712395 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fba98c-3fc9-4f3b-af27-b0c89945cf46 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:26.713100 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 336/668] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Oct 17 03:33:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:26.718340 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39854c-95c5-42a7-ad5d-2efd56e88516 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:26.823659 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39854c-95c5-42a7-ad5d-2efd56e88516 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:26.842671 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fba98c-3fc9-4f3b-af27-b0c89945cf46 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:26.864620 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63364159-d8e1-47e0-8345-a971f863f96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:26.865600 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63364159-d8e1-47e0-8345-a971f863f96f None admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:26.866121 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63364159-d8e1-47e0-8345-a971f863f96f None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:26.867017 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63364159-d8e1-47e0-8345-a971f863f96f None admin] PATH_INFO: `/v3/projects/932cdb9ef7644bf5ba57c6d1d6bce7ff/users/cecbc19355ef4570b2b05bba876a3a8b/roles/a810fe6483474b65ab4ee8969ce78455` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:26.868070 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:26.870080 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63364159-d8e1-47e0-8345-a971f863f96f None admin] RBAC: Authorizing `identity:create_grant(project_id=932cdb9ef7644bf5ba57c6d1d6bce7ff, user_id=cecbc19355ef4570b2b05bba876a3a8b, role_id=a810fe6483474b65ab4ee8969ce78455)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:26.872285 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63364159-d8e1-47e0-8345-a971f863f96f None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:26.881503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9139db67-b31f-431f-b85b-bce5a7107554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:26.882299 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139db67-b31f-431f-b85b-bce5a7107554 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:26.882653 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139db67-b31f-431f-b85b-bce5a7107554 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:26.883093 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9139db67-b31f-431f-b85b-bce5a7107554 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:26.883830 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9139db67-b31f-431f-b85b-bce5a7107554 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:26.885313 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9139db67-b31f-431f-b85b-bce5a7107554 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:26.894735 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-9139db67-b31f-431f-b85b-bce5a7107554 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 03:33:26.894735 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:26.894735 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:33:26.894735 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:33:26.894735 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:33:26.894735 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:33:26.894735 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:33:26.894735 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:26.894735 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 03:33:26.894735 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:26.894735 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:33:26.894735 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:26.894735 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:26.894735 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:26.894735 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:26.894735 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:26.899925 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:26.899925 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:26.899925 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:26.899925 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:26.899925 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:26.899925 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:26.899925 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:26.899925 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:33:26.899925 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:33:26.899925 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:33:26.899925 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:33:26.899925 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:26.899925 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:26.899925 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:26.899925 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:26.899925 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:26.899925 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:26.900819 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:26.900819 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:26.900819 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:26.900819 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:26.900819 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:26.900819 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:26.900819 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:26.900819 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:26.900819 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:26.900819 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:26.900819 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:33:26.900819 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:26.900819 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 17 03:33:26.900819 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:26.900819 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 333/669] 10.222.0.17 () {56 vars in 1132 bytes} [Mon Oct 17 03:33:26 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:33:26.903080 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9139db67-b31f-431f-b85b-bce5a7107554 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:26.959134 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 337/670] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Oct 17 03:33:26 2022] PUT /identity/v3/projects/932cdb9ef7644bf5ba57c6d1d6bce7ff/users/cecbc19355ef4570b2b05bba876a3a8b/roles/a810fe6483474b65ab4ee8969ce78455 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:33:27.021534 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9139db67-b31f-431f-b85b-bce5a7107554 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:27.062374 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb23c8c-3329-43ff-b34a-4dd6d50e5efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:27.063480 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb23c8c-3329-43ff-b34a-4dd6d50e5efe None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:27.063987 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb23c8c-3329-43ff-b34a-4dd6d50e5efe None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:27.064459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb23c8c-3329-43ff-b34a-4dd6d50e5efe None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:27.065301 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb23c8c-3329-43ff-b34a-4dd6d50e5efe None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:27.065804 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb23c8c-3329-43ff-b34a-4dd6d50e5efe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:27.067433 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb23c8c-3329-43ff-b34a-4dd6d50e5efe None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:27.074184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 334/671] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Oct 17 03:33:26 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:33:27.079620 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e91c14-6a17-45fa-9cac-e6363d73a6cb None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:27.079947 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e91c14-6a17-45fa-9cac-e6363d73a6cb None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:27.080223 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e91c14-6a17-45fa-9cac-e6363d73a6cb None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:27.081749 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 338/672] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:27.084882 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb23c8c-3329-43ff-b34a-4dd6d50e5efe None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:27.087725 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee54c6-c99a-4706-8696-ac98149bacf1 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:27.088114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee54c6-c99a-4706-8696-ac98149bacf1 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:27.088316 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feee54c6-c99a-4706-8696-ac98149bacf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:27.108880 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-feee54c6-c99a-4706-8696-ac98149bacf1 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:27.110099 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-feee54c6-c99a-4706-8696-ac98149bacf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:27.143312 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:27.149965 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:27.197562 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 339/673] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:27 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:33:27.211241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb23c8c-3329-43ff-b34a-4dd6d50e5efe None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:27.248979 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89da9d5-d8a7-466f-8377-18f5503965e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:27.249595 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89da9d5-d8a7-466f-8377-18f5503965e6 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:27.249880 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89da9d5-d8a7-466f-8377-18f5503965e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:27.250159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89da9d5-d8a7-466f-8377-18f5503965e6 None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:27.250793 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89da9d5-d8a7-466f-8377-18f5503965e6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:27.251100 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89da9d5-d8a7-466f-8377-18f5503965e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:27.252549 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89da9d5-d8a7-466f-8377-18f5503965e6 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:27.257848 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 335/674] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Oct 17 03:33:27 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:27.263671 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee54c6-c99a-4706-8696-ac98149bacf1 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:27.374609 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feee54c6-c99a-4706-8696-ac98149bacf1 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:27.408873 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66ef26b-ba77-47c8-9fef-86ff53f0a457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:27.409484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ef26b-ba77-47c8-9fef-86ff53f0a457 None admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:27.409766 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ef26b-ba77-47c8-9fef-86ff53f0a457 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:27.410038 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66ef26b-ba77-47c8-9fef-86ff53f0a457 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:27.410714 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66ef26b-ba77-47c8-9fef-86ff53f0a457 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:27.412087 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66ef26b-ba77-47c8-9fef-86ff53f0a457 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:27.470536 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89da9d5-d8a7-466f-8377-18f5503965e6 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:27.511456 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 340/675] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:33:27 2022] POST /identity/v3/endpoints => generated 347 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:33:27.592476 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89da9d5-d8a7-466f-8377-18f5503965e6 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:27.603537 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:27.649637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0bd5f0-4521-40de-a663-b8f4e5c30a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:27.649637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0bd5f0-4521-40de-a663-b8f4e5c30a44 None admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:27.649637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0bd5f0-4521-40de-a663-b8f4e5c30a44 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:27.649637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0bd5f0-4521-40de-a663-b8f4e5c30a44 None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:27.650064 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0bd5f0-4521-40de-a663-b8f4e5c30a44 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:27.650064 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0bd5f0-4521-40de-a663-b8f4e5c30a44 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:27.716920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 336/676] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Oct 17 03:33:27 2022] POST /identity/v3/services => generated 251 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:33:28.058030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ef7bb-b231-49ac-bd26-c32c028a193f None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:28.058543 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ef7bb-b231-49ac-bd26-c32c028a193f None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:28.058642 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ef7bb-b231-49ac-bd26-c32c028a193f None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:28.060138 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 341/677] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:28.065621 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54054ef-4fd4-4b54-8f41-7f60f20e6358 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:28.066044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54054ef-4fd4-4b54-8f41-7f60f20e6358 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:28.066216 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54054ef-4fd4-4b54-8f41-7f60f20e6358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:28.084663 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-b54054ef-4fd4-4b54-8f41-7f60f20e6358 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:28.085897 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-b54054ef-4fd4-4b54-8f41-7f60f20e6358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:28.161390 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 337/678] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:28 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:33:28.428777 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ef7bb-b231-49ac-bd26-c32c028a193f None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:28.455202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:28.535926 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ef7bb-b231-49ac-bd26-c32c028a193f None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:28.573471 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db9dff-079a-45ee-9852-fe79f7ddd04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:28.574082 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db9dff-079a-45ee-9852-fe79f7ddd04b None admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:28.574410 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db9dff-079a-45ee-9852-fe79f7ddd04b None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:28.574692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db9dff-079a-45ee-9852-fe79f7ddd04b None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:28.575476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db9dff-079a-45ee-9852-fe79f7ddd04b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:28.576857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db9dff-079a-45ee-9852-fe79f7ddd04b None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:28.702272 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 342/679] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Oct 17 03:33:28 2022] POST /identity/v3/services => generated 241 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:33:29.288054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d400b4-f125-4870-8e3c-4edb5f407d6e None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:29.288640 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d400b4-f125-4870-8e3c-4edb5f407d6e None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:29.288640 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d400b4-f125-4870-8e3c-4edb5f407d6e None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:29.290690 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 338/680] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:29.296100 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4418eb-90b9-462a-9aec-fffdf7b9b103 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:29.296513 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4418eb-90b9-462a-9aec-fffdf7b9b103 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:29.296704 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4418eb-90b9-462a-9aec-fffdf7b9b103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:29.315415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-1e4418eb-90b9-462a-9aec-fffdf7b9b103 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:29.316622 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-1e4418eb-90b9-462a-9aec-fffdf7b9b103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:29.394263 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 343/681] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:33:29.668148 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d400b4-f125-4870-8e3c-4edb5f407d6e None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:29.771400 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:29.784147 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c12f7-33cf-48ec-add2-ff8917fff21b None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:29.789691 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c12f7-33cf-48ec-add2-ff8917fff21b None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:29.789691 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c12f7-33cf-48ec-add2-ff8917fff21b None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:29.789901 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d400b4-f125-4870-8e3c-4edb5f407d6e None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:29.789979 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 344/682] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:33:29.794181 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82db5d-e5c6-4c0e-8ec2-e01c66794016 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:29.794727 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82db5d-e5c6-4c0e-8ec2-e01c66794016 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:29.795207 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82db5d-e5c6-4c0e-8ec2-e01c66794016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:29.814401 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-ca82db5d-e5c6-4c0e-8ec2-e01c66794016 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:29.815974 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-ca82db5d-e5c6-4c0e-8ec2-e01c66794016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:29.818186 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:29.827300 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d3ff0-ed1a-4776-8f0a-04b36b34fb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:29.828895 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d3ff0-ed1a-4776-8f0a-04b36b34fb78 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:29.828895 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d3ff0-ed1a-4776-8f0a-04b36b34fb78 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:29.828895 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8d3ff0-ed1a-4776-8f0a-04b36b34fb78 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:29.840205 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8d3ff0-ed1a-4776-8f0a-04b36b34fb78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:33:29.845194 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-3d8d3ff0-ed1a-4776-8f0a-04b36b34fb78 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 03:33:29.845194 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:29.845194 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:29.845194 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:29.845194 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:29.845194 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:29.845194 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:29.845194 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:29.845194 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:29.845194 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:29.845194 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:29.845194 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:29.845194 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:33:29.845194 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:33:29.845194 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:33:29.845194 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:33:29.846109 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:33:29.846109 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:33:29.846109 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:33:29.846109 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:33:29.846109 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:29.846109 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:29.846109 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:29.846109 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:29.846109 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:29.846109 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:29.846109 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:29.846109 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:29.846109 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:29.846109 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:29.846109 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:29.846109 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:29.847360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:29.847360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:29.847360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:29.847360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:29.847360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:33:29.847360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:33:29.847360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:33:29.847360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:33:29.847360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:33:29.847360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:33:29.847360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 03:33:29.847360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:29.847360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 339/683] 10.222.0.17 () {56 vars in 1120 bytes} [Mon Oct 17 03:33:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:29.853019 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d3ff0-ed1a-4776-8f0a-04b36b34fb78 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:29.891700 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 345/684] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:33:29.900676 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a882daa-6a3d-4af9-8d36-e8c03add7a8a None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:29.904193 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a882daa-6a3d-4af9-8d36-e8c03add7a8a None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:29.904193 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a882daa-6a3d-4af9-8d36-e8c03add7a8a None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:29.904193 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 346/685] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:29 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:29.907624 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cea69-db76-45c5-a572-dc53f49e6f94 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:29.908091 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cea69-db76-45c5-a572-dc53f49e6f94 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:29.908530 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cea69-db76-45c5-a572-dc53f49e6f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:29.930565 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-a49cea69-db76-45c5-a572-dc53f49e6f94 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:29.931657 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-a49cea69-db76-45c5-a572-dc53f49e6f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:29.978326 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8d3ff0-ed1a-4776-8f0a-04b36b34fb78 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:30.017663 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 347/686] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:29 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:33:30.019079 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecaebcb-afe9-4242-934f-08445d25ad49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:30.019889 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecaebcb-afe9-4242-934f-08445d25ad49 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:30.020328 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecaebcb-afe9-4242-934f-08445d25ad49 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:30.020777 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecaebcb-afe9-4242-934f-08445d25ad49 None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:30.021589 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecaebcb-afe9-4242-934f-08445d25ad49 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:30.022050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecaebcb-afe9-4242-934f-08445d25ad49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:30.024144 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecaebcb-afe9-4242-934f-08445d25ad49 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:30.033086 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 340/687] 10.222.0.17 () {56 vars in 1115 bytes} [Mon Oct 17 03:33:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:33:30.038578 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cea69-db76-45c5-a572-dc53f49e6f94 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:30.121111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:30.150428 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cea69-db76-45c5-a572-dc53f49e6f94 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:30.186676 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e586e7d-6b27-4d88-bda0-bcddc51e034a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:30.187513 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e586e7d-6b27-4d88-bda0-bcddc51e034a None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:30.187954 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e586e7d-6b27-4d88-bda0-bcddc51e034a None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:30.188365 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e586e7d-6b27-4d88-bda0-bcddc51e034a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:30.192200 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecaebcb-afe9-4242-934f-08445d25ad49 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:30.200023 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e586e7d-6b27-4d88-bda0-bcddc51e034a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:30.201622 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e586e7d-6b27-4d88-bda0-bcddc51e034a None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:30.208443 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:30.217319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-6e586e7d-6b27-4d88-bda0-bcddc51e034a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:30.217319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:30.217319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:30.217319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:30.217319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:30.217319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:30.217319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:30.217319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:30.217319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:30.217319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:30.217319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:30.217319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:30.217319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:30.217319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:30.217319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:30.217319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:30.224233 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:30.224233 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:30.224233 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:30.224233 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:30.224233 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:30.224233 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:30.224233 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:30.224233 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:30.224233 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:30.224233 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:30.224233 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:30.224233 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:30.224233 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:30.224233 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:30.224233 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:30.224233 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:30.224233 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:30.225109 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:30.225109 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:30.225109 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:30.225109 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 348/688] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:30.226394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e586e7d-6b27-4d88-bda0-bcddc51e034a None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:30.301631 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecaebcb-afe9-4242-934f-08445d25ad49 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:30.330070 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e586e7d-6b27-4d88-bda0-bcddc51e034a None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:30.337472 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2d075d-f5dc-4fc9-a62c-da172156a1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:30.338108 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d075d-f5dc-4fc9-a62c-da172156a1e6 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:30.338391 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d075d-f5dc-4fc9-a62c-da172156a1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:30.338664 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2d075d-f5dc-4fc9-a62c-da172156a1e6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:30.339945 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2d075d-f5dc-4fc9-a62c-da172156a1e6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:30.341315 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2d075d-f5dc-4fc9-a62c-da172156a1e6 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:30.350818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-fe2d075d-f5dc-4fc9-a62c-da172156a1e6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 03:33:30.350818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:30.350818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:33:30.350818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:33:30.350818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:33:30.350818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:33:30.350818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:33:30.350818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:30.350818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 03:33:30.350818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:30.350818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:33:30.350818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:30.350818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:30.350818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:30.350818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:30.350818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:30.350818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:30.352120 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:30.352120 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:30.352120 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:30.352120 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:30.352120 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:30.352120 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:30.352120 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:33:30.352120 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:33:30.352120 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:33:30.352120 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:33:30.352120 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:30.352120 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:30.352120 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:30.352120 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:30.352120 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:30.352120 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:30.353104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:30.353104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:30.353104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:30.353104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:30.353104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:30.353104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:30.353104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:30.353104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:30.353104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:30.353104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:30.353104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:33:30.353104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:30.353104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 03:33:30.353104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:30.353104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 341/689] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:33:30 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:33:30.354054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2d075d-f5dc-4fc9-a62c-da172156a1e6 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:30.359795 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:30.375167 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03553615-34ce-4034-a37b-174c2282e73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:30.375863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03553615-34ce-4034-a37b-174c2282e73c None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:30.376150 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03553615-34ce-4034-a37b-174c2282e73c None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:30.376428 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03553615-34ce-4034-a37b-174c2282e73c None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:30.377076 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03553615-34ce-4034-a37b-174c2282e73c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:30.377385 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03553615-34ce-4034-a37b-174c2282e73c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:30.379186 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03553615-34ce-4034-a37b-174c2282e73c None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:30.396016 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 349/690] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:30.396016 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03553615-34ce-4034-a37b-174c2282e73c None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:30.442054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:30.464914 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2d075d-f5dc-4fc9-a62c-da172156a1e6 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:30.498190 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818ce36-961c-4285-aa01-2066f40c2dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:30.498823 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818ce36-961c-4285-aa01-2066f40c2dc5 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:30.498958 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03553615-34ce-4034-a37b-174c2282e73c None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:30.499142 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818ce36-961c-4285-aa01-2066f40c2dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:30.499361 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5818ce36-961c-4285-aa01-2066f40c2dc5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:30.499964 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818ce36-961c-4285-aa01-2066f40c2dc5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:30.501330 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5818ce36-961c-4285-aa01-2066f40c2dc5 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:30.510581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-5818ce36-961c-4285-aa01-2066f40c2dc5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 03:33:30.510581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:30.510581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:33:30.510581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:33:30.510581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:33:30.510581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:33:30.510581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:33:30.510581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:30.510581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 03:33:30.510581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:30.510581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:33:30.510581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:30.510581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:30.510581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:30.510581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:30.510581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:30.510581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:30.511548 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:30.511548 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:30.511548 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:30.511548 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:30.511548 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:30.511548 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:30.511548 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:33:30.511548 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:33:30.511548 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:33:30.511548 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:33:30.511548 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:30.511548 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:30.511548 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:30.511548 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:30.511548 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:30.511548 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:30.512501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:30.512501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:30.512501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:30.512501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:30.512501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:30.512501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:30.512501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:30.512501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:30.512501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:30.512501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:30.512501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:33:30.512501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:30.512501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 03:33:30.512501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:30.512501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 342/691] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Oct 17 03:33:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:30.513721 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818ce36-961c-4285-aa01-2066f40c2dc5 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:30.540915 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09642994-91be-4eb8-9054-e278e9b7ef14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:30.541826 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09642994-91be-4eb8-9054-e278e9b7ef14 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:30.542309 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09642994-91be-4eb8-9054-e278e9b7ef14 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:30.542771 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09642994-91be-4eb8-9054-e278e9b7ef14 None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:30.543405 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09642994-91be-4eb8-9054-e278e9b7ef14 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:30.543731 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09642994-91be-4eb8-9054-e278e9b7ef14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:30.545099 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09642994-91be-4eb8-9054-e278e9b7ef14 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:30.549861 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 350/692] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Oct 17 03:33:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:30.550642 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09642994-91be-4eb8-9054-e278e9b7ef14 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:30.597607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:30.621085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5818ce36-961c-4285-aa01-2066f40c2dc5 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:30.655430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09642994-91be-4eb8-9054-e278e9b7ef14 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:30.658645 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1efdf1-083a-4047-85f7-33e899e8632f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:30.659273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1efdf1-083a-4047-85f7-33e899e8632f None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:30.659551 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1efdf1-083a-4047-85f7-33e899e8632f None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:30.659848 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1efdf1-083a-4047-85f7-33e899e8632f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:30.666921 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1efdf1-083a-4047-85f7-33e899e8632f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:30.668386 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1efdf1-083a-4047-85f7-33e899e8632f None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:30.670917 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:30.681017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-0d1efdf1-083a-4047-85f7-33e899e8632f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 03:33:30.681017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:30.681017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:30.681017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:30.681017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:30.681017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:30.681017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:30.681017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:30.681017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:30.681017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:30.681017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:30.681017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:30.681017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:33:30.681017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:33:30.681017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:33:30.681017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:33:30.684648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:30.684648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:30.684648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:33:30.684648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:30.684648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:33:30.684648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:30.684648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:30.684648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:30.684648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:30.684648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:30.684648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:30.684648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:30.684648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:30.684648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:30.684648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:30.684648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:30.684648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:30.684648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:30.686268 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:30.686268 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:30.686268 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:33:30.686268 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:33:30.686268 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:33:30.686268 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:33:30.686268 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:33:30.686268 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:33:30.686268 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 17 03:33:30.686268 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:30.686268 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 343/693] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Oct 17 03:33:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:30.686268 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1efdf1-083a-4047-85f7-33e899e8632f None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:30.703538 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b561d-c23e-4add-987f-4bc2c47befc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:30.704198 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b561d-c23e-4add-987f-4bc2c47befc7 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:30.704555 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b561d-c23e-4add-987f-4bc2c47befc7 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:30.704902 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b561d-c23e-4add-987f-4bc2c47befc7 None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:30.705575 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8b561d-c23e-4add-987f-4bc2c47befc7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:30.705850 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8b561d-c23e-4add-987f-4bc2c47befc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:30.707223 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8b561d-c23e-4add-987f-4bc2c47befc7 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:30.717251 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 351/694] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Oct 17 03:33:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:30.717251 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b561d-c23e-4add-987f-4bc2c47befc7 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:30.799218 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1efdf1-083a-4047-85f7-33e899e8632f None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:30.826301 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b561d-c23e-4add-987f-4bc2c47befc7 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:30.833815 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374a0c7-a229-4d9c-8b75-5df4dbe3f40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:30.834426 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374a0c7-a229-4d9c-8b75-5df4dbe3f40c None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:30.834708 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374a0c7-a229-4d9c-8b75-5df4dbe3f40c None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:30.834991 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4374a0c7-a229-4d9c-8b75-5df4dbe3f40c None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:30.835654 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374a0c7-a229-4d9c-8b75-5df4dbe3f40c None admin] RBAC: Adding query filter params (type=compute) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:30.835990 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374a0c7-a229-4d9c-8b75-5df4dbe3f40c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:30.837375 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4374a0c7-a229-4d9c-8b75-5df4dbe3f40c None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:30.838193 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:30.851787 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 344/695] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Oct 17 03:33:30 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:30.851787 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374a0c7-a229-4d9c-8b75-5df4dbe3f40c None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:30.883195 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa740a5-255d-45a4-8b7e-c32a5c99e454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:30.883965 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa740a5-255d-45a4-8b7e-c32a5c99e454 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:30.884247 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa740a5-255d-45a4-8b7e-c32a5c99e454 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:30.884523 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa740a5-255d-45a4-8b7e-c32a5c99e454 None admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:30.885274 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa740a5-255d-45a4-8b7e-c32a5c99e454 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:30.885579 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa740a5-255d-45a4-8b7e-c32a5c99e454 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:30.887036 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa740a5-255d-45a4-8b7e-c32a5c99e454 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:30.911790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 352/696] 10.222.0.17 () {56 vars in 1149 bytes} [Mon Oct 17 03:33:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:30.917682 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa740a5-255d-45a4-8b7e-c32a5c99e454 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:30.935062 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:30.982489 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4374a0c7-a229-4d9c-8b75-5df4dbe3f40c None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:31.021107 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d93ded-7a27-4e7d-bbba-04143ab1f7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:31.021728 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d93ded-7a27-4e7d-bbba-04143ab1f7c9 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:31.022015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d93ded-7a27-4e7d-bbba-04143ab1f7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:31.022291 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d93ded-7a27-4e7d-bbba-04143ab1f7c9 None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:31.022933 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d93ded-7a27-4e7d-bbba-04143ab1f7c9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:31.023283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d93ded-7a27-4e7d-bbba-04143ab1f7c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:31.024669 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d93ded-7a27-4e7d-bbba-04143ab1f7c9 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:31.030232 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 345/697] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Oct 17 03:33:30 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:31.031168 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b2a1a-00df-4034-be92-f632f5ed89a6 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:31.031461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b2a1a-00df-4034-be92-f632f5ed89a6 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:31.031765 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b2a1a-00df-4034-be92-f632f5ed89a6 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:31.033242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 346/698] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:31.035479 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b2a1a-00df-4034-be92-f632f5ed89a6 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:31.049293 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa740a5-255d-45a4-8b7e-c32a5c99e454 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:31.085055 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43208dca-e107-49c1-b97a-414eb974b187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:31.086188 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43208dca-e107-49c1-b97a-414eb974b187 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:31.086472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43208dca-e107-49c1-b97a-414eb974b187 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:31.086746 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43208dca-e107-49c1-b97a-414eb974b187 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:31.123996 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:31.145263 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b2a1a-00df-4034-be92-f632f5ed89a6 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:31.162061 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43208dca-e107-49c1-b97a-414eb974b187 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:31.163823 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43208dca-e107-49c1-b97a-414eb974b187 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:31.176337 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:31.185091 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4053d52-9703-47bb-b011-d284ace7da88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:31.185708 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4053d52-9703-47bb-b011-d284ace7da88 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:31.185989 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4053d52-9703-47bb-b011-d284ace7da88 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:31.186264 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4053d52-9703-47bb-b011-d284ace7da88 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:31.187019 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4053d52-9703-47bb-b011-d284ace7da88 None admin] RBAC: Adding query filter params (interface=public, service_id=7c6aa7ce883d49c693ccb5d53e6126a1, region_id=RegionOne) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:31.187333 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4053d52-9703-47bb-b011-d284ace7da88 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:31.188719 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4053d52-9703-47bb-b011-d284ace7da88 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:31.193828 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 347/699] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Oct 17 03:33:31 2022] GET /identity/v3/endpoints?service_id=7c6aa7ce883d49c693ccb5d53e6126a1&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:31.194999 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1093d3-4f4b-42df-b4b5-3c801328592a None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:31.195288 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1093d3-4f4b-42df-b4b5-3c801328592a None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:31.195563 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1093d3-4f4b-42df-b4b5-3c801328592a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:31.214534 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-3e1093d3-4f4b-42df-b4b5-3c801328592a None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:31.215729 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-3e1093d3-4f4b-42df-b4b5-3c801328592a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:31.281889 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 353/700] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:33:30 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 03:33:31.282767 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43208dca-e107-49c1-b97a-414eb974b187 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:31.287929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 348/701] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:33:31.287929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1093d3-4f4b-42df-b4b5-3c801328592a None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:31.359469 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:31.399664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43208dca-e107-49c1-b97a-414eb974b187 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:31.415833 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1093d3-4f4b-42df-b4b5-3c801328592a None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:31.439223 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:31.450665 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f92f7-17db-447b-bde8-a512b3b9b894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:31.450823 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744f78b-d834-46ae-b87f-cc11891e0db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:31.451328 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744f78b-d834-46ae-b87f-cc11891e0db0 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:31.451607 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744f78b-d834-46ae-b87f-cc11891e0db0 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:31.451910 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744f78b-d834-46ae-b87f-cc11891e0db0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:31.452071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f92f7-17db-447b-bde8-a512b3b9b894 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:31.452658 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f92f7-17db-447b-bde8-a512b3b9b894 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:31.453079 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f92f7-17db-447b-bde8-a512b3b9b894 None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:31.454018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f92f7-17db-447b-bde8-a512b3b9b894 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:31.455931 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f92f7-17db-447b-bde8-a512b3b9b894 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:31.462300 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744f78b-d834-46ae-b87f-cc11891e0db0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:31.463719 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744f78b-d834-46ae-b87f-cc11891e0db0 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:31.464997 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 354/702] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Oct 17 03:33:31 2022] GET /identity/v3/services => generated 1497 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:33:31.489398 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-b744f78b-d834-46ae-b87f-cc11891e0db0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:31.489398 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:31.489398 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:31.489398 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:31.489398 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:31.489398 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:31.489398 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:31.489398 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:31.489398 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:31.489398 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:31.489398 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:31.489398 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:31.489398 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:31.489398 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:31.489398 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:31.489398 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:31.490687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:31.490687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:31.490687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:31.490687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:31.490687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:31.490687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:31.490687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:31.490687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:31.490687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:31.490687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:31.490687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:31.490687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:31.490687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:31.490687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:31.490687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:31.490687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:31.490687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:31.491686 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:31.491686 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:31.491686 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:31.498288 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 349/703] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:31.500048 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f92f7-17db-447b-bde8-a512b3b9b894 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:31.576732 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744f78b-d834-46ae-b87f-cc11891e0db0 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:31.629004 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f92f7-17db-447b-bde8-a512b3b9b894 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:31.668771 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:31.682327 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ddff1-2b70-467f-9a35-3f1bb1eadae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:31.682959 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ddff1-2b70-467f-9a35-3f1bb1eadae1 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:31.683241 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ddff1-2b70-467f-9a35-3f1bb1eadae1 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:31.683513 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ddff1-2b70-467f-9a35-3f1bb1eadae1 None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:31.684698 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ddff1-2b70-467f-9a35-3f1bb1eadae1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:31.684999 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ddff1-2b70-467f-9a35-3f1bb1eadae1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:31.686362 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ddff1-2b70-467f-9a35-3f1bb1eadae1 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:31.704037 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 355/704] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:31.708068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ddff1-2b70-467f-9a35-3f1bb1eadae1 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:31.710314 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744f78b-d834-46ae-b87f-cc11891e0db0 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:31.825440 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ddff1-2b70-467f-9a35-3f1bb1eadae1 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:31.869322 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1861b-6ad0-4fa3-b88e-31a33e7b0280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:31.869936 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1861b-6ad0-4fa3-b88e-31a33e7b0280 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:31.870218 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1861b-6ad0-4fa3-b88e-31a33e7b0280 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:31.870500 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1861b-6ad0-4fa3-b88e-31a33e7b0280 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:31.873180 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a99755-3141-409d-a3cb-45de5d735cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:31.874066 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a99755-3141-409d-a3cb-45de5d735cc7 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:31.874594 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a99755-3141-409d-a3cb-45de5d735cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:31.875071 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a99755-3141-409d-a3cb-45de5d735cc7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:31.875878 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a99755-3141-409d-a3cb-45de5d735cc7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:31.877446 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a99755-3141-409d-a3cb-45de5d735cc7 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:31.877555 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1861b-6ad0-4fa3-b88e-31a33e7b0280 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:31.879165 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1861b-6ad0-4fa3-b88e-31a33e7b0280 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:31.887637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-53a99755-3141-409d-a3cb-45de5d735cc7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:33:31.887637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:31.887637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:33:31.887637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:33:31.887637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:33:31.887637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:33:31.887637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:33:31.887637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:31.887637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:33:31.887637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:31.887637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:33:31.887637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:31.887637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:31.887637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:31.887637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:31.887637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:31.887637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:31.888652 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:31.888652 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:31.888652 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:31.888652 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:31.888652 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:31.888652 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:31.888652 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:33:31.888652 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:33:31.888652 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:33:31.888652 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:33:31.888652 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:31.888652 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:31.888652 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:31.888652 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:31.888652 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:31.888652 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:31.889488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:31.889488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:31.889488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:31.889488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:31.889488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:31.889488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:31.889488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:31.889488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:31.889488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:31.889488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:31.889488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:33:31.889488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:31.889488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:33:31.889488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:31.892229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-16e1861b-6ad0-4fa3-b88e-31a33e7b0280 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:31.892229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:31.892229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:31.892229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:31.892229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:31.892229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:31.892229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:31.892229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:31.892229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:31.892229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:31.892229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:31.892229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:31.892229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:33:31.892229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:33:31.892229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:33:31.893060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:33:31.893060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:31.893060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:31.893060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:31.893060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:31.893060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:31.893060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:31.893060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:31.893060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:31.893060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:31.893060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:31.893060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:31.893060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:31.893060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:31.893060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:31.893060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:31.893060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:31.893917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:33:31.893917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:33:31.893917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:33:31.893917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:33:31.893917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:33:31.893917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:33:31.893917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:31.893917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:31.895122 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 350/705] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Oct 17 03:33:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:31.902688 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 356/706] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:33:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:31.902844 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a99755-3141-409d-a3cb-45de5d735cc7 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:31.904281 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1861b-6ad0-4fa3-b88e-31a33e7b0280 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:31.943507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:31.969678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:32.010508 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a99755-3141-409d-a3cb-45de5d735cc7 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:32.026905 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1861b-6ad0-4fa3-b88e-31a33e7b0280 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:32.047875 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd56361-68ed-4cc0-99e6-8b2985c9e62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:32.048826 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd56361-68ed-4cc0-99e6-8b2985c9e62a None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:32.049304 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd56361-68ed-4cc0-99e6-8b2985c9e62a None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:32.049791 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd56361-68ed-4cc0-99e6-8b2985c9e62a None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:32.050622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd56361-68ed-4cc0-99e6-8b2985c9e62a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:32.051121 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd56361-68ed-4cc0-99e6-8b2985c9e62a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:32.052725 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd56361-68ed-4cc0-99e6-8b2985c9e62a None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:32.058411 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 351/707] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Oct 17 03:33:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:32.063640 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd56361-68ed-4cc0-99e6-8b2985c9e62a None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:32.070729 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062160cb-7761-441f-b558-d8d7584032e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:32.071345 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062160cb-7761-441f-b558-d8d7584032e7 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:32.071627 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062160cb-7761-441f-b558-d8d7584032e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:32.072033 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062160cb-7761-441f-b558-d8d7584032e7 None admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:32.072619 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062160cb-7761-441f-b558-d8d7584032e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:32.072968 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062160cb-7761-441f-b558-d8d7584032e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:32.074807 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062160cb-7761-441f-b558-d8d7584032e7 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:32.090591 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 357/708] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Oct 17 03:33:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:33:32.097037 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062160cb-7761-441f-b558-d8d7584032e7 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:32.177589 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd56361-68ed-4cc0-99e6-8b2985c9e62a None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:32.184051 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:32.212023 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062160cb-7761-441f-b558-d8d7584032e7 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:32.225314 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:32.229075 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8328e2-121e-4f3b-b66c-2dc8aef4673a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:32.229849 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8328e2-121e-4f3b-b66c-2dc8aef4673a None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:32.230135 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8328e2-121e-4f3b-b66c-2dc8aef4673a None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:32.230408 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8328e2-121e-4f3b-b66c-2dc8aef4673a None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:32.231462 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8328e2-121e-4f3b-b66c-2dc8aef4673a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:32.231807 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8328e2-121e-4f3b-b66c-2dc8aef4673a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:32.233690 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8328e2-121e-4f3b-b66c-2dc8aef4673a None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:32.243773 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 352/709] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Oct 17 03:33:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:33:32.246543 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8328e2-121e-4f3b-b66c-2dc8aef4673a None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:32.259551 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523e1f5-7fdd-4895-b09e-1086f36a3c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:32.260319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523e1f5-7fdd-4895-b09e-1086f36a3c97 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:32.260698 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523e1f5-7fdd-4895-b09e-1086f36a3c97 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:32.262262 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523e1f5-7fdd-4895-b09e-1086f36a3c97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:32.263126 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523e1f5-7fdd-4895-b09e-1086f36a3c97 None admin] RBAC: Adding query filter params (role.id=a810fe6483474b65ab4ee8969ce78455, scope.project.id=932cdb9ef7644bf5ba57c6d1d6bce7ff, user.id=cecbc19355ef4570b2b05bba876a3a8b) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:32.263829 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523e1f5-7fdd-4895-b09e-1086f36a3c97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:32.265648 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8523e1f5-7fdd-4895-b09e-1086f36a3c97 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:32.275776 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 358/710] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Oct 17 03:33:32 2022] GET /identity/v3/role_assignments?user.id=cecbc19355ef4570b2b05bba876a3a8b&scope.project.id=932cdb9ef7644bf5ba57c6d1d6bce7ff&role.id=a810fe6483474b65ab4ee8969ce78455 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:33:32.366177 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8328e2-121e-4f3b-b66c-2dc8aef4673a None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:32.405086 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860675eb-4f38-4569-ac48-4c23655ec2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:32.406141 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860675eb-4f38-4569-ac48-4c23655ec2ad None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:32.406641 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860675eb-4f38-4569-ac48-4c23655ec2ad None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:32.407133 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860675eb-4f38-4569-ac48-4c23655ec2ad None admin] PATH_INFO: `/v3/endpoints` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:32.408144 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860675eb-4f38-4569-ac48-4c23655ec2ad None admin] RBAC: Adding query filter params (interface=public, service_id=c4bd5661c6654c83a9e189f63a16ff74, region_id=RegionOne) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:32.408686 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860675eb-4f38-4569-ac48-4c23655ec2ad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:32.410432 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860675eb-4f38-4569-ac48-4c23655ec2ad None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:32.417286 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 353/711] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Oct 17 03:33:32 2022] GET /identity/v3/endpoints?service_id=c4bd5661c6654c83a9e189f63a16ff74&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:33:32.424141 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523e1f5-7fdd-4895-b09e-1086f36a3c97 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:32.532392 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523e1f5-7fdd-4895-b09e-1086f36a3c97 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:32.566701 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f45800-5ba9-45c9-9774-ced56c168864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:32.567454 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f45800-5ba9-45c9-9774-ced56c168864 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:32.567865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f45800-5ba9-45c9-9774-ced56c168864 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:32.568235 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f45800-5ba9-45c9-9774-ced56c168864 None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:32.568940 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f45800-5ba9-45c9-9774-ced56c168864 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:32.570964 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f45800-5ba9-45c9-9774-ced56c168864 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:32.577567 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 359/712] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Oct 17 03:33:32 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:33:33.045120 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89321f5b-081b-44e2-b51d-b66e48a0e6fd None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:33.045507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89321f5b-081b-44e2-b51d-b66e48a0e6fd None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:33.045705 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89321f5b-081b-44e2-b51d-b66e48a0e6fd None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:33.047232 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 354/713] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:33.052633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e36c7c-1761-4453-90cc-e8b014b41bf1 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:33.052917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e36c7c-1761-4453-90cc-e8b014b41bf1 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:33.053189 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e36c7c-1761-4453-90cc-e8b014b41bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:33.072077 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-61e36c7c-1761-4453-90cc-e8b014b41bf1 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:33.073256 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-61e36c7c-1761-4453-90cc-e8b014b41bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:33.104751 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:33.143775 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 360/714] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:33:33.413922 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89321f5b-081b-44e2-b51d-b66e48a0e6fd None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:33.446628 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:33.523246 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89321f5b-081b-44e2-b51d-b66e48a0e6fd None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:33.558517 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ddf3b-1624-4cf4-a5ec-69775356a47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:33.559124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ddf3b-1624-4cf4-a5ec-69775356a47c None admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:33.559400 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ddf3b-1624-4cf4-a5ec-69775356a47c None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:33.559668 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ddf3b-1624-4cf4-a5ec-69775356a47c None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:33.560356 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ddf3b-1624-4cf4-a5ec-69775356a47c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:33.561722 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ddf3b-1624-4cf4-a5ec-69775356a47c None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:33.635661 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 355/715] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Oct 17 03:33:33 2022] POST /identity/v3/services => generated 242 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:33:33.693271 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac51e4-f8c7-4d1c-96cc-963289e32040 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:33.693799 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac51e4-f8c7-4d1c-96cc-963289e32040 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:33.693799 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac51e4-f8c7-4d1c-96cc-963289e32040 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:33.696127 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 361/716] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:33.701518 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39300b-3375-4061-aa5d-57057fdb7dbd None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:33.701801 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39300b-3375-4061-aa5d-57057fdb7dbd None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:33.702072 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39300b-3375-4061-aa5d-57057fdb7dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:33.722205 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-dd39300b-3375-4061-aa5d-57057fdb7dbd None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:33.723354 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-dd39300b-3375-4061-aa5d-57057fdb7dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:33.809945 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 356/717] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:33 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:33:34.076361 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac51e4-f8c7-4d1c-96cc-963289e32040 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:34.187702 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac51e4-f8c7-4d1c-96cc-963289e32040 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:34.223197 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1368fd0-ace3-4f4b-a60a-6e05ff7b6965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:34.223927 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1368fd0-ace3-4f4b-a60a-6e05ff7b6965 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:34.224250 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1368fd0-ace3-4f4b-a60a-6e05ff7b6965 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:34.224527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1368fd0-ace3-4f4b-a60a-6e05ff7b6965 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:34.225101 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1368fd0-ace3-4f4b-a60a-6e05ff7b6965 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:34.226489 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1368fd0-ace3-4f4b-a60a-6e05ff7b6965 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:34.235566 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-d1368fd0-ace3-4f4b-a60a-6e05ff7b6965 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 03:33:34.235566 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:34.235566 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:33:34.235566 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:33:34.235566 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:33:34.235566 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:33:34.235566 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:33:34.235566 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:34.235566 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 03:33:34.235566 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:34.235566 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:33:34.235566 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:34.235566 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:34.235566 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:34.235566 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:34.235566 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:34.235566 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:34.236580 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:34.236580 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:34.236580 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:34.236580 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:34.236580 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:34.236580 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:34.236580 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:33:34.236580 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:33:34.236580 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:33:34.236580 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:33:34.236580 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:34.236580 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:34.236580 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:34.236580 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:34.236580 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:34.236580 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:34.237564 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:34.237564 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:34.237564 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:34.237564 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:34.237564 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:34.237564 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:34.237564 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:34.237564 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:34.237564 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:34.237564 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:34.237564 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:33:34.237564 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:34.237564 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 17 03:33:34.237564 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:34.237564 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 362/718] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Oct 17 03:33:34 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:34.243206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39300b-3375-4061-aa5d-57057fdb7dbd None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:34.260934 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:34.343851 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39300b-3375-4061-aa5d-57057fdb7dbd None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:34.383692 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1fff0-025a-46d6-8d1e-29081643816e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:34.384344 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1fff0-025a-46d6-8d1e-29081643816e None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:34.384621 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1fff0-025a-46d6-8d1e-29081643816e None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:34.384891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af1fff0-025a-46d6-8d1e-29081643816e None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:34.385531 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1fff0-025a-46d6-8d1e-29081643816e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:34.385834 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1fff0-025a-46d6-8d1e-29081643816e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:34.387199 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af1fff0-025a-46d6-8d1e-29081643816e None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:34.392846 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 357/719] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Oct 17 03:33:34 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:34.398229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1368fd0-ace3-4f4b-a60a-6e05ff7b6965 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:34.432404 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:34.507273 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1368fd0-ace3-4f4b-a60a-6e05ff7b6965 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:34.541402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1382f-b120-4c36-a05c-99882605e0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:34.542079 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1382f-b120-4c36-a05c-99882605e0ba None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:34.542361 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1382f-b120-4c36-a05c-99882605e0ba None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:34.542631 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1382f-b120-4c36-a05c-99882605e0ba None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:34.543259 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f1382f-b120-4c36-a05c-99882605e0ba None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:34.543554 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f1382f-b120-4c36-a05c-99882605e0ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:34.544924 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f1382f-b120-4c36-a05c-99882605e0ba None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:34.550176 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 363/720] 10.222.0.17 () {56 vars in 1114 bytes} [Mon Oct 17 03:33:34 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:34.556424 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1fff0-025a-46d6-8d1e-29081643816e None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:34.603852 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e2d4f-cd39-4498-8480-383cbe4e9c21 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:34.604095 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e2d4f-cd39-4498-8480-383cbe4e9c21 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:34.604321 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e2d4f-cd39-4498-8480-383cbe4e9c21 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:34.605826 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 364/721] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:33:34.611895 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c3b37-b43e-4fe4-98e9-b006b028bacf None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:34.612177 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c3b37-b43e-4fe4-98e9-b006b028bacf None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:34.612454 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c3b37-b43e-4fe4-98e9-b006b028bacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:34.636358 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-b03c3b37-b43e-4fe4-98e9-b006b028bacf None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:34.637707 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-b03c3b37-b43e-4fe4-98e9-b006b028bacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:34.651391 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:34.671999 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af1fff0-025a-46d6-8d1e-29081643816e None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:34.709998 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b2c32-c410-4314-9141-c54ac8555815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:34.711137 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b2c32-c410-4314-9141-c54ac8555815 None admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:34.711637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b2c32-c410-4314-9141-c54ac8555815 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:34.712161 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b2c32-c410-4314-9141-c54ac8555815 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:34.713200 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b2c32-c410-4314-9141-c54ac8555815 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:34.715095 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b2c32-c410-4314-9141-c54ac8555815 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:34.729124 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 365/722] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:34 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:33:34.801072 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 358/723] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:33:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:33:34.833361 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853175bf-74d4-4c71-8a65-b551bf832758 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:34.833635 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853175bf-74d4-4c71-8a65-b551bf832758 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:34.833879 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853175bf-74d4-4c71-8a65-b551bf832758 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:34.836094 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 366/724] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:34.842222 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740997d-79fc-4edd-8854-ef4da644788d None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:34.842784 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740997d-79fc-4edd-8854-ef4da644788d None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:34.843239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740997d-79fc-4edd-8854-ef4da644788d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:34.864216 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-0740997d-79fc-4edd-8854-ef4da644788d None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:34.865774 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-0740997d-79fc-4edd-8854-ef4da644788d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:34.946854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 359/725] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:34 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:33:35.014813 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853175bf-74d4-4c71-8a65-b551bf832758 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:35.043451 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:35.118363 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853175bf-74d4-4c71-8a65-b551bf832758 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:35.151891 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887fc256-af72-4595-b427-809de6ccb982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:35.152525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887fc256-af72-4595-b427-809de6ccb982 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:35.152805 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887fc256-af72-4595-b427-809de6ccb982 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:35.153079 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887fc256-af72-4595-b427-809de6ccb982 None admin] PATH_INFO: `/v3/services/image` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:35.153712 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887fc256-af72-4595-b427-809de6ccb982 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:35.155125 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887fc256-af72-4595-b427-809de6ccb982 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:35.164527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-887fc256-af72-4595-b427-809de6ccb982 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 03:33:35.164527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:35.164527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:33:35.164527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:33:35.164527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:33:35.164527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:33:35.164527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:33:35.164527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:35.164527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 03:33:35.164527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:35.164527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:33:35.164527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:35.164527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:35.164527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:35.164527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:35.164527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:35.164527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:35.165506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:35.165506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:35.165506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:35.165506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:35.165506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:35.165506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:35.165506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:33:35.165506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:33:35.165506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:33:35.165506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:33:35.165506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:35.165506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:35.165506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:35.165506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:35.165506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:35.165506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:35.166328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:35.166328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:35.166328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:35.166328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:35.166328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:35.166328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:35.166328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:35.166328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:35.166328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:35.166328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:35.166328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:33:35.166328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:35.166328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 03:33:35.166328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:35.166328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 367/726] 10.222.0.17 () {56 vars in 1105 bytes} [Mon Oct 17 03:33:35 2022] GET /identity/v3/services/image => generated 86 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:35.171769 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740997d-79fc-4edd-8854-ef4da644788d None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:35.228805 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887fc256-af72-4595-b427-809de6ccb982 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:35.239403 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:35.291787 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740997d-79fc-4edd-8854-ef4da644788d None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:35.327922 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3d75f-d609-4d9f-8ab2-7e4b1652bdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:35.328620 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3d75f-d609-4d9f-8ab2-7e4b1652bdc7 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:35.328984 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3d75f-d609-4d9f-8ab2-7e4b1652bdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:35.329320 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3d75f-d609-4d9f-8ab2-7e4b1652bdc7 None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:35.330055 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b3d75f-d609-4d9f-8ab2-7e4b1652bdc7 None admin] RBAC: Adding query filter params (name=image) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:35.330413 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b3d75f-d609-4d9f-8ab2-7e4b1652bdc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:35.331784 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b3d75f-d609-4d9f-8ab2-7e4b1652bdc7 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:35.337577 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 360/727] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:35.337909 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:35.343499 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3d75f-d609-4d9f-8ab2-7e4b1652bdc7 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:35.353197 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887fc256-af72-4595-b427-809de6ccb982 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:35.393302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59723d76-4b37-4faa-8f96-5e66c8514aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:35.393861 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59723d76-4b37-4faa-8f96-5e66c8514aa5 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:35.394190 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59723d76-4b37-4faa-8f96-5e66c8514aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:35.394470 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59723d76-4b37-4faa-8f96-5e66c8514aa5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:35.395058 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59723d76-4b37-4faa-8f96-5e66c8514aa5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:35.397020 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59723d76-4b37-4faa-8f96-5e66c8514aa5 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:35.408442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-59723d76-4b37-4faa-8f96-5e66c8514aa5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:33:35.408442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:35.408442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:33:35.408442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:33:35.408442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:33:35.408442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:33:35.408442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:33:35.408442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:35.408442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:33:35.408442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:35.408442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:33:35.408442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:35.408442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:35.408442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:35.408442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:35.408442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:35.408442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:35.409463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:35.409463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:35.409463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:35.409463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:35.409463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:35.409463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:35.409463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:33:35.409463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:33:35.409463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:33:35.409463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:33:35.409463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:35.409463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:35.409463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:35.409463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:35.409463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:35.409463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:35.410372 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:35.410372 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:35.410372 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:35.410372 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:35.410372 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:35.410372 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:35.410372 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:35.410372 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:35.410372 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:35.410372 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:35.410372 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:33:35.410372 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:35.410372 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:33:35.410372 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:35.414583 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 368/728] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Oct 17 03:33:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:33:35.416000 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59723d76-4b37-4faa-8f96-5e66c8514aa5 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:35.478555 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3d75f-d609-4d9f-8ab2-7e4b1652bdc7 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:35.514190 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:35.527817 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3afe7-20f5-45c8-9408-774ed650ee18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:35.528733 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3afe7-20f5-45c8-9408-774ed650ee18 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:35.529215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3afe7-20f5-45c8-9408-774ed650ee18 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:35.529411 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59723d76-4b37-4faa-8f96-5e66c8514aa5 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:35.529883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3afe7-20f5-45c8-9408-774ed650ee18 None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:35.530959 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3afe7-20f5-45c8-9408-774ed650ee18 None admin] RBAC: Adding query filter params (type=image) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:35.531474 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3afe7-20f5-45c8-9408-774ed650ee18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:35.533206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3afe7-20f5-45c8-9408-774ed650ee18 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:35.548293 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 361/729] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:35 2022] GET /identity/v3/services?type=image => generated 122 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:33:35.577601 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1926ef-50a2-4517-8bd3-22771b232683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:35.578237 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1926ef-50a2-4517-8bd3-22771b232683 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:35.578589 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1926ef-50a2-4517-8bd3-22771b232683 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:35.578869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1926ef-50a2-4517-8bd3-22771b232683 None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:35.579568 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1926ef-50a2-4517-8bd3-22771b232683 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:35.579912 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1926ef-50a2-4517-8bd3-22771b232683 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:35.581352 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1926ef-50a2-4517-8bd3-22771b232683 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:35.587962 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 369/730] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Oct 17 03:33:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:35.594149 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3afe7-20f5-45c8-9408-774ed650ee18 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:35.720949 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3afe7-20f5-45c8-9408-774ed650ee18 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:35.759132 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62560e87-41dd-4fab-bcb4-c1b2efc4a1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:35.760204 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62560e87-41dd-4fab-bcb4-c1b2efc4a1cf None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:35.760697 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62560e87-41dd-4fab-bcb4-c1b2efc4a1cf None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:35.761200 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62560e87-41dd-4fab-bcb4-c1b2efc4a1cf None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:35.762069 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62560e87-41dd-4fab-bcb4-c1b2efc4a1cf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:35.762610 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62560e87-41dd-4fab-bcb4-c1b2efc4a1cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:35.764261 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62560e87-41dd-4fab-bcb4-c1b2efc4a1cf None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:35.770650 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 362/731] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Oct 17 03:33:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:35.776718 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1926ef-50a2-4517-8bd3-22771b232683 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:35.797902 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:35.888838 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1926ef-50a2-4517-8bd3-22771b232683 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:35.927266 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c50c81-f2d9-40ae-8286-9a241cb0b893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:35.927907 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c50c81-f2d9-40ae-8286-9a241cb0b893 None admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:35.928188 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c50c81-f2d9-40ae-8286-9a241cb0b893 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:35.928456 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c50c81-f2d9-40ae-8286-9a241cb0b893 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:35.929711 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c50c81-f2d9-40ae-8286-9a241cb0b893 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:35.931056 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c50c81-f2d9-40ae-8286-9a241cb0b893 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:35.953806 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b4010-510f-4481-9f00-c5d26499434f None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:35.954054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b4010-510f-4481-9f00-c5d26499434f None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:35.954289 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b4010-510f-4481-9f00-c5d26499434f None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:35.956755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 363/732] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:35.960884 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd97a31-8b30-40dc-98e8-f2b947d01b3b None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:35.961169 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd97a31-8b30-40dc-98e8-f2b947d01b3b None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:35.961447 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd97a31-8b30-40dc-98e8-f2b947d01b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:35.980677 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-bfd97a31-8b30-40dc-98e8-f2b947d01b3b None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:35.981988 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-bfd97a31-8b30-40dc-98e8-f2b947d01b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:36.018391 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 370/733] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:33:35 2022] POST /identity/v3/endpoints => generated 349 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:33:36.039156 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:36.071922 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 364/734] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:35 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:33:36.343031 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c50c81-f2d9-40ae-8286-9a241cb0b893 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:36.422957 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:36.448028 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c50c81-f2d9-40ae-8286-9a241cb0b893 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:36.484675 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c8fa2-fdfa-4979-8739-acb73d53002b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:36.485316 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c8fa2-fdfa-4979-8739-acb73d53002b None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:36.485601 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c8fa2-fdfa-4979-8739-acb73d53002b None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:36.485872 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c8fa2-fdfa-4979-8739-acb73d53002b None admin] PATH_INFO: `/v3/services/compute` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:36.486510 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c8fa2-fdfa-4979-8739-acb73d53002b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:36.487939 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c8fa2-fdfa-4979-8739-acb73d53002b None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:36.497736 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-077c8fa2-fdfa-4979-8739-acb73d53002b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 03:33:36.497736 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:36.497736 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:33:36.497736 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:33:36.497736 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:33:36.497736 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:33:36.497736 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:33:36.497736 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:36.497736 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 03:33:36.497736 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:36.497736 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:33:36.497736 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:36.497736 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:36.497736 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:36.497736 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:36.497736 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:36.497736 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:36.498785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:36.498785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:36.498785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:36.498785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:36.498785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:36.498785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:36.498785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:33:36.498785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:33:36.498785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:33:36.498785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:33:36.498785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:36.498785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:36.498785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:36.498785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:36.498785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:36.498785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:36.499611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:36.499611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:36.499611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:36.499611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:36.499611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:36.499611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:36.499611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:36.499611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:36.499611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:36.499611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:36.499611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:33:36.499611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:36.499611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 03:33:36.499611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:36.503890 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 371/735] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:33:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:36.505810 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd97a31-8b30-40dc-98e8-f2b947d01b3b None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:36.609471 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd97a31-8b30-40dc-98e8-f2b947d01b3b None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:36.649006 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5101ad0-fb82-46c8-9af6-f21b4df5ddc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:36.649619 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5101ad0-fb82-46c8-9af6-f21b4df5ddc7 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:36.649903 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5101ad0-fb82-46c8-9af6-f21b4df5ddc7 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:36.650173 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5101ad0-fb82-46c8-9af6-f21b4df5ddc7 None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:36.650846 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5101ad0-fb82-46c8-9af6-f21b4df5ddc7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:36.651143 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5101ad0-fb82-46c8-9af6-f21b4df5ddc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:36.652531 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5101ad0-fb82-46c8-9af6-f21b4df5ddc7 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:36.657801 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 365/736] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Oct 17 03:33:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:36.665300 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c8fa2-fdfa-4979-8739-acb73d53002b None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:36.770631 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c8fa2-fdfa-4979-8739-acb73d53002b None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:36.805645 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ace30-fbe8-44da-928c-49cdde2c3b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:36.806268 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ace30-fbe8-44da-928c-49cdde2c3b8a None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:36.806525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ace30-fbe8-44da-928c-49cdde2c3b8a None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:36.806798 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ace30-fbe8-44da-928c-49cdde2c3b8a None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:36.807427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ace30-fbe8-44da-928c-49cdde2c3b8a None admin] RBAC: Adding query filter params (type=compute) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:36.807730 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ace30-fbe8-44da-928c-49cdde2c3b8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:36.809109 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ace30-fbe8-44da-928c-49cdde2c3b8a None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:36.819137 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 372/737] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Oct 17 03:33:36 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:36.821065 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5101ad0-fb82-46c8-9af6-f21b4df5ddc7 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:36.869754 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:36.926176 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5101ad0-fb82-46c8-9af6-f21b4df5ddc7 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:36.961519 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50af63aa-02e9-4394-a18e-e86b498f77bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:36.962188 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af63aa-02e9-4394-a18e-e86b498f77bb None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:36.962470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af63aa-02e9-4394-a18e-e86b498f77bb None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:36.962741 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50af63aa-02e9-4394-a18e-e86b498f77bb None admin] PATH_INFO: `/v3/endpoints` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:36.963488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af63aa-02e9-4394-a18e-e86b498f77bb None admin] RBAC: Adding query filter params (interface=public, service_id=4348e893bc074afe8e12a6ca2cdf9d0d, region_id=RegionOne) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:36.964089 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af63aa-02e9-4394-a18e-e86b498f77bb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:36.965439 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50af63aa-02e9-4394-a18e-e86b498f77bb None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:36.970628 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 366/738] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Oct 17 03:33:36 2022] GET /identity/v3/endpoints?service_id=4348e893bc074afe8e12a6ca2cdf9d0d&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:33:36.976728 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ace30-fbe8-44da-928c-49cdde2c3b8a None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:36.980130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:37.091574 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ace30-fbe8-44da-928c-49cdde2c3b8a None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:37.110001 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85c079-d5f0-41f0-9971-f9ba1e800a54 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:37.110573 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85c079-d5f0-41f0-9971-f9ba1e800a54 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:37.111039 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85c079-d5f0-41f0-9971-f9ba1e800a54 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:37.112829 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 367/739] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:33:37.118540 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc70f84-5b0c-48c0-9f11-784cc293e604 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:37.119052 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc70f84-5b0c-48c0-9f11-784cc293e604 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:37.119527 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc70f84-5b0c-48c0-9f11-784cc293e604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:37.127064 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd149312-1556-4575-ba61-73186044e524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:37.127612 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd149312-1556-4575-ba61-73186044e524 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:37.127928 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd149312-1556-4575-ba61-73186044e524 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:37.128205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd149312-1556-4575-ba61-73186044e524 None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:37.128810 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd149312-1556-4575-ba61-73186044e524 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:37.130243 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd149312-1556-4575-ba61-73186044e524 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:37.140303 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-efc70f84-5b0c-48c0-9f11-784cc293e604 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:37.141808 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-efc70f84-5b0c-48c0-9f11-784cc293e604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:37.147780 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 373/740] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Oct 17 03:33:36 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:33:37.226880 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 368/741] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:33:37.491119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd149312-1556-4575-ba61-73186044e524 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:37.561985 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:37.596586 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd149312-1556-4575-ba61-73186044e524 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:37.632059 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f314b-44f9-4046-9bda-5be113f73df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:37.632668 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f314b-44f9-4046-9bda-5be113f73df7 None admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:37.632945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f314b-44f9-4046-9bda-5be113f73df7 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:37.633207 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f314b-44f9-4046-9bda-5be113f73df7 None admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:37.633843 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097f314b-44f9-4046-9bda-5be113f73df7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:37.635249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097f314b-44f9-4046-9bda-5be113f73df7 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:37.637863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: WARNING keystone.server.flask.common [None req-097f314b-44f9-4046-9bda-5be113f73df7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 17 03:33:37.705222 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.sql.core [None req-097f314b-44f9-4046-9bda-5be113f73df7 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 17 03:33:37.705222 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 03:33:37.705222 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [parameters: {'id': '6d3d03b8718f4e2b9eaa2801ecd495d2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 17 03:33:37.705222 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88300) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 03:33:37.712682 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-097f314b-44f9-4046-9bda-5be113f73df7 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6d3d03b8718f4e2b9eaa2801ecd495d2.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6d3d03b8718f4e2b9eaa2801ecd495d2. Oct 17 03:33:37.712682 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:37.712682 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 03:33:37.712682 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 03:33:37.712682 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 03:33:37.712682 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 03:33:37.712682 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 03:33:37.712682 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 03:33:37.712682 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 03:33:37.712682 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application conn.query(q) Oct 17 03:33:37.712682 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 03:33:37.712682 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 03:33:37.712682 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 03:33:37.713629 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application result.read() Oct 17 03:33:37.713629 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 03:33:37.713629 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 03:33:37.713629 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 03:33:37.713629 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 03:33:37.713629 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 03:33:37.713629 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 03:33:37.713629 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 03:33:37.713629 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 03:33:37.713629 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 17 03:33:37.713629 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:37.713629 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 03:33:37.713629 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:37.713629 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:37.713629 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 03:33:37.713629 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 03:33:37.713629 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 17 03:33:37.713629 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 17 03:33:37.714842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 03:33:37.714842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application next(self.gen) Oct 17 03:33:37.714842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 03:33:37.714842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application yield resource Oct 17 03:33:37.714842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 03:33:37.714842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application next(self.gen) Oct 17 03:33:37.714842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 03:33:37.714842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 03:33:37.714842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 03:33:37.714842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 03:33:37.714842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 03:33:37.714842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise self.value Oct 17 03:33:37.714842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 03:33:37.714842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 03:33:37.714842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 03:33:37.714842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application session.commit() Oct 17 03:33:37.714842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 03:33:37.717001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 03:33:37.717001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 03:33:37.717001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 03:33:37.717001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 03:33:37.717001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self.session.flush() Oct 17 03:33:37.717001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 03:33:37.717001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 03:33:37.717001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 03:33:37.717001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 03:33:37.717001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 03:33:37.717001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application compat.raise_( Oct 17 03:33:37.717001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 03:33:37.717001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception Oct 17 03:33:37.717001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 03:33:37.717001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 03:33:37.717001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 03:33:37.717001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 03:33:37.719133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 03:33:37.719133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 03:33:37.719133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 03:33:37.719133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 03:33:37.719133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 17 03:33:37.719133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 17 03:33:37.719133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 03:33:37.719133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 03:33:37.719133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 03:33:37.719133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 03:33:37.719133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 03:33:37.719133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 03:33:37.719133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 03:33:37.719133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 03:33:37.719133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 03:33:37.720161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 03:33:37.720161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 03:33:37.720161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception Oct 17 03:33:37.720161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 03:33:37.720161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 03:33:37.720161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 03:33:37.720161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 03:33:37.720161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 03:33:37.720161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 03:33:37.720161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 03:33:37.720161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application conn.query(q) Oct 17 03:33:37.720161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 03:33:37.720161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 03:33:37.720161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 03:33:37.720161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application result.read() Oct 17 03:33:37.720161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 03:33:37.720161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 03:33:37.721231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 03:33:37.721231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 03:33:37.721231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 03:33:37.721231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 03:33:37.721231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 03:33:37.721231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 03:33:37.721231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 17 03:33:37.721231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 03:33:37.721231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [parameters: {'id': '6d3d03b8718f4e2b9eaa2801ecd495d2', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 17 03:33:37.721231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 03:33:37.721231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:37.721231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:33:37.721231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:37.721231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:37.721231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 17 03:33:37.722135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 17 03:33:37.722135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 03:33:37.722135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 03:33:37.722135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 17 03:33:37.722135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:37.722135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:33:37.722135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:37.722135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:37.722135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:37.722135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:37.722135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:37.722135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:37.722135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:37.722135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:37.722135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:37.722135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:37.722135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:37.723222 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:37.723222 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 17 03:33:37.723222 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 17 03:33:37.723222 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:37.723222 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:37.723222 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 17 03:33:37.723222 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 17 03:33:37.723222 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6d3d03b8718f4e2b9eaa2801ecd495d2. Oct 17 03:33:37.723222 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:37.723222 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 374/742] 10.222.0.17 () {60 vars in 1140 bytes} [Mon Oct 17 03:33:37 2022] POST /identity/v3/projects => generated 262 bytes in 224 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:37.728006 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc70f84-5b0c-48c0-9f11-784cc293e604 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:37.767626 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:37.810080 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e16d02-91d8-444a-a824-77bf51c56817 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:37.810591 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e16d02-91d8-444a-a824-77bf51c56817 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:37.810667 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e16d02-91d8-444a-a824-77bf51c56817 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:37.815095 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 375/743] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:37.820415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a8e91-058d-485d-a1f6-33efc74ef665 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:37.820705 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a8e91-058d-485d-a1f6-33efc74ef665 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:37.820985 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a8e91-058d-485d-a1f6-33efc74ef665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:37.832708 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc70f84-5b0c-48c0-9f11-784cc293e604 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:37.842618 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-918a8e91-058d-485d-a1f6-33efc74ef665 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:37.844294 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-918a8e91-058d-485d-a1f6-33efc74ef665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:37.873210 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72704bab-e6e3-47a8-b6ad-06dbdc281eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:37.874312 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72704bab-e6e3-47a8-b6ad-06dbdc281eff None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:37.874605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72704bab-e6e3-47a8-b6ad-06dbdc281eff None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:37.875159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72704bab-e6e3-47a8-b6ad-06dbdc281eff None admin] PATH_INFO: `/v3/projects/service` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:37.884457 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72704bab-e6e3-47a8-b6ad-06dbdc281eff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:37.886236 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72704bab-e6e3-47a8-b6ad-06dbdc281eff None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:37.902621 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-72704bab-e6e3-47a8-b6ad-06dbdc281eff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:37.902621 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:37.902621 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:37.902621 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:37.902621 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:37.902621 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:37.902621 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:37.902621 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:37.902621 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:37.902621 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:37.902621 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:37.902621 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:37.902621 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:33:37.902621 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:33:37.902621 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:33:37.903542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:33:37.903542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:37.903542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:37.903542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:37.903542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:37.903542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:37.903542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:37.903542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:37.903542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:37.903542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:37.903542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:37.903542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:37.903542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:37.903542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:37.903542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:37.903542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:37.903542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:37.904449 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:33:37.904449 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:33:37.904449 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:33:37.904449 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:33:37.904449 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:33:37.904449 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:33:37.904449 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:37.904449 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:37.904449 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 369/744] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:33:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:37.910521 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72704bab-e6e3-47a8-b6ad-06dbdc281eff None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:37.927820 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 376/745] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:33:38.018627 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:38.023423 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72704bab-e6e3-47a8-b6ad-06dbdc281eff None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:38.059417 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab5f97-98b0-45fc-b16f-a2641439739a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:38.060455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab5f97-98b0-45fc-b16f-a2641439739a None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:38.060967 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab5f97-98b0-45fc-b16f-a2641439739a None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:38.061437 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab5f97-98b0-45fc-b16f-a2641439739a None admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:38.062358 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab5f97-98b0-45fc-b16f-a2641439739a None admin] RBAC: Adding query filter params (name=service) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:38.062866 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab5f97-98b0-45fc-b16f-a2641439739a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:38.064579 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab5f97-98b0-45fc-b16f-a2641439739a None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:38.083134 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 370/746] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Oct 17 03:33:37 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:33:38.206634 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a8e91-058d-485d-a1f6-33efc74ef665 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:38.268975 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:38.318761 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a8e91-058d-485d-a1f6-33efc74ef665 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:38.328429 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5e851-673b-468b-94ee-ea84b48bd57f None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:38.328710 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5e851-673b-468b-94ee-ea84b48bd57f None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:38.328978 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5e851-673b-468b-94ee-ea84b48bd57f None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:38.330454 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 371/747] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:33:38.336000 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee382fc-4e06-4ca4-baea-ae76f0e22558 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:38.336425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee382fc-4e06-4ca4-baea-ae76f0e22558 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:38.336596 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee382fc-4e06-4ca4-baea-ae76f0e22558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:38.356779 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-6ee382fc-4e06-4ca4-baea-ae76f0e22558 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:38.357985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6ee382fc-4e06-4ca4-baea-ae76f0e22558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:38.360747 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d653a107-83fa-4b60-8e4f-d5a9f44078fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:38.361354 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653a107-83fa-4b60-8e4f-d5a9f44078fb None admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:38.361632 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653a107-83fa-4b60-8e4f-d5a9f44078fb None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:38.361905 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653a107-83fa-4b60-8e4f-d5a9f44078fb None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:38.362552 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d653a107-83fa-4b60-8e4f-d5a9f44078fb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:38.364077 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d653a107-83fa-4b60-8e4f-d5a9f44078fb None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:38.431474 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 372/748] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:38 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:33:38.446740 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 377/749] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Oct 17 03:33:38 2022] POST /identity/v3/services => generated 242 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:33:38.725138 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee382fc-4e06-4ca4-baea-ae76f0e22558 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:38.820130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:38.832961 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee382fc-4e06-4ca4-baea-ae76f0e22558 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:38.875640 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8482bc-40c6-4d4e-9038-67a647bb09e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:38.876669 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8482bc-40c6-4d4e-9038-67a647bb09e3 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:38.877273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8482bc-40c6-4d4e-9038-67a647bb09e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:38.877737 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8482bc-40c6-4d4e-9038-67a647bb09e3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:38.878499 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8482bc-40c6-4d4e-9038-67a647bb09e3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:38.880135 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8482bc-40c6-4d4e-9038-67a647bb09e3 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:38.889455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-6b8482bc-40c6-4d4e-9038-67a647bb09e3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:33:38.889455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:38.889455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:33:38.889455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:33:38.889455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:33:38.889455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:33:38.889455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:33:38.889455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:38.889455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:33:38.889455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:38.889455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:33:38.889455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:38.889455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:38.889455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:38.889455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:38.889455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:38.889455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:38.890453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:38.890453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:38.890453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:38.890453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:38.890453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:38.890453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:38.890453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:33:38.890453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:33:38.890453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:33:38.890453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:33:38.890453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:38.890453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:38.890453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:38.890453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:38.890453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:38.890453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:38.891372 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:38.891372 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:38.891372 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:38.891372 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:38.891372 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:38.891372 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:38.891372 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:38.891372 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:38.891372 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:38.891372 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:38.891372 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:33:38.891372 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:38.891372 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:33:38.891372 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:38.893814 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 373/750] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Oct 17 03:33:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:38.900646 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d653a107-83fa-4b60-8e4f-d5a9f44078fb None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:39.005707 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d653a107-83fa-4b60-8e4f-d5a9f44078fb None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:39.024449 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:39.046078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec79e03-5c90-4b7a-afeb-2f27c438f41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:39.046686 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec79e03-5c90-4b7a-afeb-2f27c438f41f None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:39.046965 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec79e03-5c90-4b7a-afeb-2f27c438f41f None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:39.047232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec79e03-5c90-4b7a-afeb-2f27c438f41f None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:39.047881 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec79e03-5c90-4b7a-afeb-2f27c438f41f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:39.048182 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec79e03-5c90-4b7a-afeb-2f27c438f41f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:39.049532 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec79e03-5c90-4b7a-afeb-2f27c438f41f None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:39.054791 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 378/751] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Oct 17 03:33:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:39.060526 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8482bc-40c6-4d4e-9038-67a647bb09e3 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:39.169830 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8482bc-40c6-4d4e-9038-67a647bb09e3 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:39.205861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3adc826-e9d4-4dde-875a-e49ba5489aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:39.206822 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3adc826-e9d4-4dde-875a-e49ba5489aa8 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:39.207301 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3adc826-e9d4-4dde-875a-e49ba5489aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:39.207802 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3adc826-e9d4-4dde-875a-e49ba5489aa8 None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:39.208646 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3adc826-e9d4-4dde-875a-e49ba5489aa8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:39.209142 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3adc826-e9d4-4dde-875a-e49ba5489aa8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:39.210742 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3adc826-e9d4-4dde-875a-e49ba5489aa8 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:39.220116 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 374/752] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Oct 17 03:33:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:39.223002 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec79e03-5c90-4b7a-afeb-2f27c438f41f None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:39.333251 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec79e03-5c90-4b7a-afeb-2f27c438f41f None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:39.365250 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de059c3d-4905-464c-a4e5-2987727865d5 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:39.365552 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de059c3d-4905-464c-a4e5-2987727865d5 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:39.365805 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de059c3d-4905-464c-a4e5-2987727865d5 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:39.367405 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 375/753] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:39.368165 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d8bbe-4f52-4e48-839a-bea1c4bfe103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:39.368767 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d8bbe-4f52-4e48-839a-bea1c4bfe103 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:39.369098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d8bbe-4f52-4e48-839a-bea1c4bfe103 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:39.369374 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d8bbe-4f52-4e48-839a-bea1c4bfe103 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:39.370127 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d8bbe-4f52-4e48-839a-bea1c4bfe103 None admin] RBAC: Adding query filter params (interface=admin, service_id=c4bd5661c6654c83a9e189f63a16ff74, region_id=RegionOne) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:39.370424 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d8bbe-4f52-4e48-839a-bea1c4bfe103 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:39.371825 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d8bbe-4f52-4e48-839a-bea1c4bfe103 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:39.372459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e540f-c7c3-4d0a-bd0c-8620ede8ba9a None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:39.372747 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e540f-c7c3-4d0a-bd0c-8620ede8ba9a None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:39.373028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1e540f-c7c3-4d0a-bd0c-8620ede8ba9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:39.379906 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 379/754] 10.222.0.17 () {56 vars in 1249 bytes} [Mon Oct 17 03:33:39 2022] GET /identity/v3/endpoints?service_id=c4bd5661c6654c83a9e189f63a16ff74&interface=admin®ion_id=RegionOne => generated 193 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:39.383213 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d8bbe-4f52-4e48-839a-bea1c4bfe103 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:39.395590 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-fa1e540f-c7c3-4d0a-bd0c-8620ede8ba9a None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:39.396943 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-fa1e540f-c7c3-4d0a-bd0c-8620ede8ba9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:39.406145 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:39.427944 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:39.498778 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d8bbe-4f52-4e48-839a-bea1c4bfe103 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:39.500120 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 376/755] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:39 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:33:39.534150 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb80442-3db8-4498-b6f5-6b5e0398d2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:39.535004 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb80442-3db8-4498-b6f5-6b5e0398d2f1 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:39.535457 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb80442-3db8-4498-b6f5-6b5e0398d2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:39.536021 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb80442-3db8-4498-b6f5-6b5e0398d2f1 None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:39.536790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb80442-3db8-4498-b6f5-6b5e0398d2f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:39.538918 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb80442-3db8-4498-b6f5-6b5e0398d2f1 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:39.545967 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 380/756] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Oct 17 03:33:39 2022] GET /identity/v3/services => generated 1957 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:33:39.785293 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e540f-c7c3-4d0a-bd0c-8620ede8ba9a None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:39.892504 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1e540f-c7c3-4d0a-bd0c-8620ede8ba9a None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:39.929322 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c849987-0072-4e8b-bda0-e5dc40b870f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:39.930330 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c849987-0072-4e8b-bda0-e5dc40b870f9 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:39.930807 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c849987-0072-4e8b-bda0-e5dc40b870f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:39.931344 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c849987-0072-4e8b-bda0-e5dc40b870f9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:39.932147 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c849987-0072-4e8b-bda0-e5dc40b870f9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:39.933689 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c849987-0072-4e8b-bda0-e5dc40b870f9 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:39.942845 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-1c849987-0072-4e8b-bda0-e5dc40b870f9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 03:33:39.942845 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:39.942845 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:33:39.942845 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:33:39.942845 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:33:39.942845 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:33:39.942845 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:33:39.942845 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:39.942845 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 03:33:39.942845 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:39.942845 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:33:39.942845 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:39.942845 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:39.942845 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:39.942845 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:39.942845 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:39.942845 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:39.943840 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:39.943840 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:39.943840 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:39.943840 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:39.943840 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:39.943840 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:39.943840 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:33:39.943840 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:33:39.943840 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:33:39.943840 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:33:39.943840 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:39.943840 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:39.943840 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:39.943840 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:39.943840 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:39.943840 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:39.945215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:39.945215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:39.945215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:39.945215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:39.945215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:39.945215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:39.945215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:39.945215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:39.945215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:39.945215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:39.945215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:33:39.945215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:39.945215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 17 03:33:39.945215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:39.951846 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 377/757] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:33:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:39.954181 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb80442-3db8-4498-b6f5-6b5e0398d2f1 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:40.060383 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb80442-3db8-4498-b6f5-6b5e0398d2f1 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:40.063068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:40.097044 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869087d-5c6d-4f3f-b80d-f990dd014557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:40.098061 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869087d-5c6d-4f3f-b80d-f990dd014557 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:40.098607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869087d-5c6d-4f3f-b80d-f990dd014557 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:40.099089 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869087d-5c6d-4f3f-b80d-f990dd014557 None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:40.099951 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869087d-5c6d-4f3f-b80d-f990dd014557 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:40.100447 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869087d-5c6d-4f3f-b80d-f990dd014557 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:40.102026 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869087d-5c6d-4f3f-b80d-f990dd014557 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:40.111884 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 381/758] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Oct 17 03:33:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:40.113866 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c849987-0072-4e8b-bda0-e5dc40b870f9 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:40.124311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:40.219977 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c849987-0072-4e8b-bda0-e5dc40b870f9 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:40.257611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd1b825-4357-4511-8c11-be75b34b26a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:40.258656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1b825-4357-4511-8c11-be75b34b26a3 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:40.259146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1b825-4357-4511-8c11-be75b34b26a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:40.259628 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd1b825-4357-4511-8c11-be75b34b26a3 None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:40.260526 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd1b825-4357-4511-8c11-be75b34b26a3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:40.261058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd1b825-4357-4511-8c11-be75b34b26a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:40.262673 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd1b825-4357-4511-8c11-be75b34b26a3 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:40.273969 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 378/759] 10.222.0.17 () {56 vars in 1112 bytes} [Mon Oct 17 03:33:40 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:40.276008 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869087d-5c6d-4f3f-b80d-f990dd014557 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:40.371999 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f2921-c0ae-4ede-85cb-9eb069688fb9 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:40.374331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f2921-c0ae-4ede-85cb-9eb069688fb9 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:40.374703 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f2921-c0ae-4ede-85cb-9eb069688fb9 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:40.377346 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 379/760] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:40.382776 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c54c6-0f61-4799-985a-94de909b8320 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:40.383347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c54c6-0f61-4799-985a-94de909b8320 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:40.383855 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c54c6-0f61-4799-985a-94de909b8320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:40.385680 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869087d-5c6d-4f3f-b80d-f990dd014557 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:40.402441 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-0b2c54c6-0f61-4799-985a-94de909b8320 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:40.404018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-0b2c54c6-0f61-4799-985a-94de909b8320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:40.429283 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da4d44-08fd-4837-8bb0-b14a8b59c0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:40.430131 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da4d44-08fd-4837-8bb0-b14a8b59c0a4 None admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:40.430410 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da4d44-08fd-4837-8bb0-b14a8b59c0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:40.430678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da4d44-08fd-4837-8bb0-b14a8b59c0a4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:40.431854 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da4d44-08fd-4837-8bb0-b14a8b59c0a4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:40.433662 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da4d44-08fd-4837-8bb0-b14a8b59c0a4 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:40.473438 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:40.478904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 380/761] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:40 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:33:40.519905 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 382/762] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:33:40 2022] POST /identity/v3/endpoints => generated 334 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:33:40.754951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c54c6-0f61-4799-985a-94de909b8320 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:40.770168 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce4ad2-534b-42fe-80b7-5a7510a5f63d None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:40.770489 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce4ad2-534b-42fe-80b7-5a7510a5f63d None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:40.770799 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce4ad2-534b-42fe-80b7-5a7510a5f63d None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:40.772777 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 383/763] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:33:40.778604 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602b155-ed57-4bb9-b159-13582fde508f None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:40.778887 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602b155-ed57-4bb9-b159-13582fde508f None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:40.779164 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602b155-ed57-4bb9-b159-13582fde508f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:40.801611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-d602b155-ed57-4bb9-b159-13582fde508f None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:40.802820 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-d602b155-ed57-4bb9-b159-13582fde508f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:40.821772 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:40.872876 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c54c6-0f61-4799-985a-94de909b8320 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:40.885849 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 384/764] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:40 2022] POST /identity/v3/auth/tokens => generated 1822 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:33:40.915836 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c870a2b-82dc-4dbd-ae52-b3bca57bbd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:40.916475 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c870a2b-82dc-4dbd-ae52-b3bca57bbd19 None admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:40.916721 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c870a2b-82dc-4dbd-ae52-b3bca57bbd19 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:40.916998 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c870a2b-82dc-4dbd-ae52-b3bca57bbd19 None admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:40.917651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c870a2b-82dc-4dbd-ae52-b3bca57bbd19 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:40.919073 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c870a2b-82dc-4dbd-ae52-b3bca57bbd19 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:40.921649 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: WARNING keystone.server.flask.common [None req-9c870a2b-82dc-4dbd-ae52-b3bca57bbd19 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 17 03:33:40.965281 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.sql.core [None req-9c870a2b-82dc-4dbd-ae52-b3bca57bbd19 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 17 03:33:40.965281 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 17 03:33:40.965281 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [parameters: {'user_id': 'c20912f56bad461f84a90bf36bf2b82a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 17 03:33:40.965281 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88299) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 03:33:40.983238 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-9c870a2b-82dc-4dbd-ae52-b3bca57bbd19 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 17 03:33:40.983238 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:40.983238 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 03:33:40.983238 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 03:33:40.983238 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 03:33:40.983238 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 03:33:40.983238 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 03:33:40.983238 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 03:33:40.983238 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 03:33:40.983238 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application conn.query(q) Oct 17 03:33:40.983238 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 03:33:40.983238 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 03:33:40.983238 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 03:33:40.983238 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application result.read() Oct 17 03:33:40.983238 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 03:33:40.985713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 03:33:40.985713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 03:33:40.985713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 03:33:40.985713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 03:33:40.985713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 03:33:40.985713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 03:33:40.985713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 03:33:40.985713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 17 03:33:40.985713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:40.985713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 03:33:40.985713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:40.985713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:40.985713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 03:33:40.985713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 03:33:40.985713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 17 03:33:40.985713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 17 03:33:40.985713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 03:33:40.985713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application next(self.gen) Oct 17 03:33:40.986611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 03:33:40.986611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application yield resource Oct 17 03:33:40.986611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 03:33:40.986611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application next(self.gen) Oct 17 03:33:40.986611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 03:33:40.986611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 03:33:40.986611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 03:33:40.986611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 03:33:40.986611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 03:33:40.986611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise self.value Oct 17 03:33:40.986611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 03:33:40.986611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 03:33:40.986611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 03:33:40.986611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application session.commit() Oct 17 03:33:40.986611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 03:33:40.986611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 03:33:40.986611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 03:33:40.987473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 03:33:40.987473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 03:33:40.987473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application self.session.flush() Oct 17 03:33:40.987473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 03:33:40.987473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 03:33:40.987473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 03:33:40.987473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 03:33:40.987473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 03:33:40.987473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application compat.raise_( Oct 17 03:33:40.987473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 03:33:40.987473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception Oct 17 03:33:40.987473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 03:33:40.987473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 03:33:40.987473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 03:33:40.987473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 03:33:40.987473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 03:33:40.987473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 03:33:40.989605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 03:33:40.989605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 03:33:40.989605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 17 03:33:40.989605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 17 03:33:40.989605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 03:33:40.989605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 03:33:40.989605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 03:33:40.989605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 03:33:40.989605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 03:33:40.989605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 03:33:40.989605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 03:33:40.989605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 03:33:40.989605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 03:33:40.989605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 03:33:40.989605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 03:33:40.989605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception Oct 17 03:33:40.995353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 03:33:40.995353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 03:33:40.995353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 03:33:40.995353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 03:33:40.995353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 03:33:40.995353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 03:33:40.995353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 03:33:40.995353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application conn.query(q) Oct 17 03:33:40.995353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 03:33:40.995353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 03:33:40.995353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 03:33:40.995353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application result.read() Oct 17 03:33:40.995353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 03:33:40.995353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 03:33:40.995353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 03:33:40.995353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 03:33:40.997461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 03:33:40.997461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 03:33:40.997461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 03:33:40.997461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 03:33:40.997461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 17 03:33:40.997461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 17 03:33:40.997461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c20912f56bad461f84a90bf36bf2b82a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 17 03:33:40.997461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 03:33:40.997461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:40.997461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:33:40.997461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:40.997461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:40.997461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:40.997461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:40.997461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:40.999309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:40.999309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:40.999309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:40.999309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:40.999309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:40.999309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:40.999309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:40.999309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 17 03:33:40.999309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 17 03:33:40.999309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:40.999309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:40.999309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:33:40.999309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:40.999309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:33:40.999309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:40.999309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 17 03:33:40.999309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 17 03:33:41.002340 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 17 03:33:41.002340 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 17 03:33:41.002340 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 03:33:41.002340 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 03:33:41.002340 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 17 03:33:41.002340 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:41.002340 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 381/765] 10.222.0.17 () {60 vars in 1131 bytes} [Mon Oct 17 03:33:40 2022] POST /identity/v3/users => generated 160 bytes in 230 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:41.003211 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602b155-ed57-4bb9-b159-13582fde508f None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:41.103121 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602b155-ed57-4bb9-b159-13582fde508f None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:41.138213 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85aaf00-d3be-4230-9fc3-3dcd5194e1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:41.138854 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85aaf00-d3be-4230-9fc3-3dcd5194e1be None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:41.139133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85aaf00-d3be-4230-9fc3-3dcd5194e1be None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:41.139405 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85aaf00-d3be-4230-9fc3-3dcd5194e1be None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:41.146780 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85aaf00-d3be-4230-9fc3-3dcd5194e1be None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:41.148755 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85aaf00-d3be-4230-9fc3-3dcd5194e1be None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:41.160458 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-c85aaf00-d3be-4230-9fc3-3dcd5194e1be None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 03:33:41.160458 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:41.160458 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:41.160458 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:41.160458 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:41.160458 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:41.160458 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:41.160458 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:41.160458 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:41.160458 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:41.160458 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:41.160458 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:41.160458 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:33:41.160458 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:33:41.160458 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:33:41.160458 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:33:41.161397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:41.161397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:41.161397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:33:41.161397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:41.161397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:33:41.161397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:41.161397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:41.161397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:41.161397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:41.161397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:41.161397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:41.161397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:41.161397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:41.161397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:41.161397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:41.161397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:41.161397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:41.161397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:41.162328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:41.162328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:41.162328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:33:41.162328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:33:41.162328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:33:41.162328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:33:41.162328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:33:41.162328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:33:41.162328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 17 03:33:41.162328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:41.162328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 385/766] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:33:40 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:41.168432 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c870a2b-82dc-4dbd-ae52-b3bca57bbd19 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:41.203078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85aaf00-d3be-4230-9fc3-3dcd5194e1be None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:41.217713 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:41.235891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:41.288990 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c870a2b-82dc-4dbd-ae52-b3bca57bbd19 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:41.321639 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85aaf00-d3be-4230-9fc3-3dcd5194e1be None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:41.341517 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444af13-96cc-414e-8ad1-319ffba59413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:41.342185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444af13-96cc-414e-8ad1-319ffba59413 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:41.342474 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444af13-96cc-414e-8ad1-319ffba59413 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:41.342751 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e444af13-96cc-414e-8ad1-319ffba59413 None admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:41.344058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444af13-96cc-414e-8ad1-319ffba59413 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:41.344375 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444af13-96cc-414e-8ad1-319ffba59413 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:41.346590 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e444af13-96cc-414e-8ad1-319ffba59413 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:41.358392 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc82fc0-d626-4d22-b2a3-15612757219a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:41.359093 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc82fc0-d626-4d22-b2a3-15612757219a None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:41.359461 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc82fc0-d626-4d22-b2a3-15612757219a None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:41.359852 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc82fc0-d626-4d22-b2a3-15612757219a None admin] PATH_INFO: `/v3/services/image` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:41.360545 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc82fc0-d626-4d22-b2a3-15612757219a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:41.362042 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc82fc0-d626-4d22-b2a3-15612757219a None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:41.371287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-2fc82fc0-d626-4d22-b2a3-15612757219a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 03:33:41.371287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:41.371287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:33:41.371287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:33:41.371287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:33:41.371287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:33:41.371287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:33:41.371287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:41.371287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 03:33:41.371287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:41.371287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:33:41.371287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:41.371287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:41.371287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:41.371287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:41.371287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:41.371287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:41.372670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:41.372670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:41.372670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:41.372670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:41.372670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:41.372670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:41.372670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:33:41.372670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:33:41.372670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:33:41.372670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:33:41.372670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:41.372670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:41.372670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:41.372670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:41.372670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:41.372670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:41.374486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:41.374486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:41.374486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:41.374486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:41.374486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:41.374486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:41.374486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:41.374486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:41.374486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:41.374486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:41.374486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:33:41.374486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:41.374486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 03:33:41.374486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:41.383769 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 386/767] 10.222.0.17 () {56 vars in 1105 bytes} [Mon Oct 17 03:33:41 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:33:41.383769 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc82fc0-d626-4d22-b2a3-15612757219a None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:41.395466 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 382/768] 10.222.0.17 () {56 vars in 1101 bytes} [Mon Oct 17 03:33:41 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:41.475396 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:41.503830 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc82fc0-d626-4d22-b2a3-15612757219a None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:41.546393 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501e407-887d-4ead-a2e3-434c28bead8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:41.547071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501e407-887d-4ead-a2e3-434c28bead8a None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:41.547357 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501e407-887d-4ead-a2e3-434c28bead8a None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:41.547630 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501e407-887d-4ead-a2e3-434c28bead8a None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:41.548309 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8501e407-887d-4ead-a2e3-434c28bead8a None admin] RBAC: Adding query filter params (name=image) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:41.548613 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8501e407-887d-4ead-a2e3-434c28bead8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:41.549984 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8501e407-887d-4ead-a2e3-434c28bead8a None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:41.556015 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 387/769] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:41.561819 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444af13-96cc-414e-8ad1-319ffba59413 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:41.685623 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e444af13-96cc-414e-8ad1-319ffba59413 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:41.716643 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:41.726514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94f820-1b4e-4484-b6c3-c008e3f06574 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:41.734796 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94f820-1b4e-4484-b6c3-c008e3f06574 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:41.734796 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94f820-1b4e-4484-b6c3-c008e3f06574 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:41.734796 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 388/770] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:41.735034 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d25d7f-69fa-4eba-aafb-494a7e584308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:41.735034 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d25d7f-69fa-4eba-aafb-494a7e584308 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:41.735034 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d25d7f-69fa-4eba-aafb-494a7e584308 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:41.735034 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d25d7f-69fa-4eba-aafb-494a7e584308 None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:41.735304 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d25d7f-69fa-4eba-aafb-494a7e584308 None admin] RBAC: Adding query filter params (type=image) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:41.735304 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d25d7f-69fa-4eba-aafb-494a7e584308 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:41.735304 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d25d7f-69fa-4eba-aafb-494a7e584308 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:41.735486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1c010-b321-4df4-bc64-1d2901686642 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:41.735486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1c010-b321-4df4-bc64-1d2901686642 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:41.735486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1c010-b321-4df4-bc64-1d2901686642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:41.740765 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 383/771] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:41.746306 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d25d7f-69fa-4eba-aafb-494a7e584308 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:41.754663 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-f1b1c010-b321-4df4-bc64-1d2901686642 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:41.755946 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-f1b1c010-b321-4df4-bc64-1d2901686642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:41.828986 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 389/772] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:41 2022] POST /identity/v3/auth/tokens => generated 1822 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:33:41.853861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d25d7f-69fa-4eba-aafb-494a7e584308 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:41.888239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e7ba0-a8e7-4f4b-a50d-aa5f4b7346ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:41.888887 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e7ba0-a8e7-4f4b-a50d-aa5f4b7346ad None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:41.889177 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e7ba0-a8e7-4f4b-a50d-aa5f4b7346ad None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:41.889453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e7ba0-a8e7-4f4b-a50d-aa5f4b7346ad None admin] PATH_INFO: `/v3/endpoints` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:41.890150 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e7ba0-a8e7-4f4b-a50d-aa5f4b7346ad None admin] RBAC: Adding query filter params (interface=public, service_id=5e5b6875a24f4d4395b5bc68f2380e38, region_id=RegionOne) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:41.890462 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e7ba0-a8e7-4f4b-a50d-aa5f4b7346ad None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:41.891855 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e7ba0-a8e7-4f4b-a50d-aa5f4b7346ad None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:41.899891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 384/773] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Oct 17 03:33:41 2022] GET /identity/v3/endpoints?service_id=5e5b6875a24f4d4395b5bc68f2380e38&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:33:41.903025 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1c010-b321-4df4-bc64-1d2901686642 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:41.968977 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:42.002925 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1c010-b321-4df4-bc64-1d2901686642 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:42.038756 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1be2fd-cfc6-4240-a82f-7ecbd8ef70d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:42.039400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1be2fd-cfc6-4240-a82f-7ecbd8ef70d5 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:42.039634 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1be2fd-cfc6-4240-a82f-7ecbd8ef70d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:42.039930 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1be2fd-cfc6-4240-a82f-7ecbd8ef70d5 None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:42.040527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1be2fd-cfc6-4240-a82f-7ecbd8ef70d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:42.041869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1be2fd-cfc6-4240-a82f-7ecbd8ef70d5 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:42.056054 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 390/774] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Oct 17 03:33:41 2022] GET /identity/v3/services => generated 1957 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:33:42.097983 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e7ba0-a8e7-4f4b-a50d-aa5f4b7346ad None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:42.152366 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:42.207388 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e7ba0-a8e7-4f4b-a50d-aa5f4b7346ad None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:42.246256 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0bff3f-8d95-45a2-b1eb-bc752b00c304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:42.246818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0bff3f-8d95-45a2-b1eb-bc752b00c304 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:42.247100 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0bff3f-8d95-45a2-b1eb-bc752b00c304 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:42.247380 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0bff3f-8d95-45a2-b1eb-bc752b00c304 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:42.247985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0bff3f-8d95-45a2-b1eb-bc752b00c304 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:42.249353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0bff3f-8d95-45a2-b1eb-bc752b00c304 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:42.258257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-be0bff3f-8d95-45a2-b1eb-bc752b00c304 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:33:42.258257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:42.258257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:33:42.258257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:33:42.258257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:33:42.258257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:33:42.258257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:33:42.258257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:42.258257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:33:42.258257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:42.258257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:33:42.258257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:42.258257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:42.258257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:42.258257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:42.258257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:42.258257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:42.259202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:42.259202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:42.259202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:42.259202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:42.259202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:42.259202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:42.259202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:33:42.259202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:33:42.259202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:33:42.259202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:33:42.259202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:42.259202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:42.259202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:42.259202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:42.259202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:42.259202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:42.260099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:42.260099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:42.260099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:42.260099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:42.260099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:42.260099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:42.260099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:42.260099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:42.260099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:42.260099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:42.260099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:33:42.260099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:42.260099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 17 03:33:42.260099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:42.260099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 385/775] 10.222.0.17 () {56 vars in 1126 bytes} [Mon Oct 17 03:33:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:42.265667 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1be2fd-cfc6-4240-a82f-7ecbd8ef70d5 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:42.365700 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1be2fd-cfc6-4240-a82f-7ecbd8ef70d5 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:42.395029 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:42.400986 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edeca0b-a76e-4beb-83d9-a2a1c836ac3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:42.401601 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edeca0b-a76e-4beb-83d9-a2a1c836ac3d None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:42.401882 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edeca0b-a76e-4beb-83d9-a2a1c836ac3d None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:42.402152 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edeca0b-a76e-4beb-83d9-a2a1c836ac3d None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:42.402779 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edeca0b-a76e-4beb-83d9-a2a1c836ac3d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:42.403083 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edeca0b-a76e-4beb-83d9-a2a1c836ac3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:42.404529 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edeca0b-a76e-4beb-83d9-a2a1c836ac3d None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:42.414686 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 391/776] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Oct 17 03:33:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:42.415802 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0bff3f-8d95-45a2-b1eb-bc752b00c304 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:42.515845 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0bff3f-8d95-45a2-b1eb-bc752b00c304 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:42.531478 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:42.553211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea35615-eb47-44b4-88be-ed13fa6a5315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:42.553843 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea35615-eb47-44b4-88be-ed13fa6a5315 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:42.554125 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea35615-eb47-44b4-88be-ed13fa6a5315 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:42.554399 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea35615-eb47-44b4-88be-ed13fa6a5315 None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:42.555026 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea35615-eb47-44b4-88be-ed13fa6a5315 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:42.555329 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea35615-eb47-44b4-88be-ed13fa6a5315 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:42.556723 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea35615-eb47-44b4-88be-ed13fa6a5315 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:42.566415 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 386/777] 10.222.0.17 () {56 vars in 1122 bytes} [Mon Oct 17 03:33:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:33:42.567536 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edeca0b-a76e-4beb-83d9-a2a1c836ac3d None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:42.666224 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edeca0b-a76e-4beb-83d9-a2a1c836ac3d None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:42.707210 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c5df6d-0016-4a68-ac65-27dd8c54c9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:42.707869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5df6d-0016-4a68-ac65-27dd8c54c9e3 None admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:42.708153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5df6d-0016-4a68-ac65-27dd8c54c9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:42.708424 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c5df6d-0016-4a68-ac65-27dd8c54c9e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:42.709107 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c5df6d-0016-4a68-ac65-27dd8c54c9e3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:42.711017 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c5df6d-0016-4a68-ac65-27dd8c54c9e3 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:42.817880 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 392/778] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:33:42 2022] POST /identity/v3/endpoints => generated 325 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:33:42.946362 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f2b4f-3b82-41b4-8eb0-211f4522895f None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:42.946739 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f2b4f-3b82-41b4-8eb0-211f4522895f None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:42.946916 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817f2b4f-3b82-41b4-8eb0-211f4522895f None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:42.948552 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 387/779] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:42.954145 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155c223-32be-42b2-9b54-b33cdca2e73b None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:42.954437 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155c223-32be-42b2-9b54-b33cdca2e73b None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:42.954707 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7155c223-32be-42b2-9b54-b33cdca2e73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:42.974400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-7155c223-32be-42b2-9b54-b33cdca2e73b None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:42.975658 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-7155c223-32be-42b2-9b54-b33cdca2e73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:43.028663 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:43.051101 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 393/780] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:42 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:33:43.314930 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f2b4f-3b82-41b4-8eb0-211f4522895f None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:43.416264 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817f2b4f-3b82-41b4-8eb0-211f4522895f None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:43.450212 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edab4e-ed64-4248-adc5-bcb8c05ee968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:43.450836 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edab4e-ed64-4248-adc5-bcb8c05ee968 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:43.451112 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edab4e-ed64-4248-adc5-bcb8c05ee968 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:43.451377 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edab4e-ed64-4248-adc5-bcb8c05ee968 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:43.461342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3edab4e-ed64-4248-adc5-bcb8c05ee968 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:33:43.466444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-d3edab4e-ed64-4248-adc5-bcb8c05ee968 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 03:33:43.466444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:43.466444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:43.466444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:43.466444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:43.466444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:43.466444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:43.466444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:43.466444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:43.466444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:43.466444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:43.466444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:43.466444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:33:43.466444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:33:43.466444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:33:43.466444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:33:43.467341 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:33:43.467341 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:33:43.467341 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:33:43.467341 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:33:43.467341 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:43.467341 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:43.467341 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:43.467341 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:43.467341 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:43.467341 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:43.467341 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:43.467341 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:43.467341 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:43.467341 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:43.467341 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:43.467341 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:43.468244 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:43.468244 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:43.468244 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:43.468244 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:43.468244 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:33:43.468244 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:33:43.468244 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:33:43.468244 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:33:43.468244 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:33:43.468244 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:33:43.468244 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 03:33:43.468244 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:43.472070 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 388/781] 10.222.0.17 () {56 vars in 1120 bytes} [Mon Oct 17 03:33:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:43.473478 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155c223-32be-42b2-9b54-b33cdca2e73b None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:43.571775 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7155c223-32be-42b2-9b54-b33cdca2e73b None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:43.605179 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0490c-6f48-4350-b2e6-a7868972b67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:43.605790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0490c-6f48-4350-b2e6-a7868972b67a None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:43.606067 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0490c-6f48-4350-b2e6-a7868972b67a None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:43.606341 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0490c-6f48-4350-b2e6-a7868972b67a None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:43.606999 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0490c-6f48-4350-b2e6-a7868972b67a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:43.607298 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0490c-6f48-4350-b2e6-a7868972b67a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:43.608687 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0490c-6f48-4350-b2e6-a7868972b67a None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:43.621939 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 394/782] 10.222.0.17 () {56 vars in 1115 bytes} [Mon Oct 17 03:33:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:43.623175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edab4e-ed64-4248-adc5-bcb8c05ee968 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:43.660200 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:43.730510 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edab4e-ed64-4248-adc5-bcb8c05ee968 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:43.766846 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889af05-557d-453e-a14a-6046eef30948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:43.767480 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889af05-557d-453e-a14a-6046eef30948 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:43.767811 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889af05-557d-453e-a14a-6046eef30948 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:43.768175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889af05-557d-453e-a14a-6046eef30948 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:43.778315 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6889af05-557d-453e-a14a-6046eef30948 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:43.779759 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6889af05-557d-453e-a14a-6046eef30948 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:43.794656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-6889af05-557d-453e-a14a-6046eef30948 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:43.794656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:43.794656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:43.794656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:43.794656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:43.794656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:43.794656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:43.794656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:43.794656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:43.794656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:43.794656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:43.794656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:43.794656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:43.794656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:43.794656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:43.794656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:43.795584 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:43.795584 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:43.795584 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:43.795584 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:43.795584 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:43.795584 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:43.795584 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:43.795584 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:43.795584 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:43.795584 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:43.795584 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:43.795584 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:43.795584 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:43.795584 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:43.795584 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:43.795584 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:43.795584 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:43.801093 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:43.801093 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:43.801093 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:43.801093 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 389/783] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:43.801772 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0490c-6f48-4350-b2e6-a7868972b67a None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:43.849561 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:43.904005 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0490c-6f48-4350-b2e6-a7868972b67a None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:43.937126 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbcefc7-2b69-4b85-8a13-0312e8ff7538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:43.937730 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbcefc7-2b69-4b85-8a13-0312e8ff7538 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:43.938010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbcefc7-2b69-4b85-8a13-0312e8ff7538 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:43.938280 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbcefc7-2b69-4b85-8a13-0312e8ff7538 None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:43.938919 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbcefc7-2b69-4b85-8a13-0312e8ff7538 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:43.939220 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbcefc7-2b69-4b85-8a13-0312e8ff7538 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:43.940583 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbcefc7-2b69-4b85-8a13-0312e8ff7538 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:43.956822 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 395/784] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:33:43.957997 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889af05-557d-453e-a14a-6046eef30948 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:44.048398 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:44.060444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6889af05-557d-453e-a14a-6046eef30948 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:44.095610 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30f3f9e-54d8-4f73-8d8a-a97d84e6ad1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:44.096270 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f3f9e-54d8-4f73-8d8a-a97d84e6ad1d None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:44.096549 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f3f9e-54d8-4f73-8d8a-a97d84e6ad1d None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:44.096822 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30f3f9e-54d8-4f73-8d8a-a97d84e6ad1d None admin] PATH_INFO: `/v3/users/nova` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:44.104788 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f3f9e-54d8-4f73-8d8a-a97d84e6ad1d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:44.106159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30f3f9e-54d8-4f73-8d8a-a97d84e6ad1d None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:44.119028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-e30f3f9e-54d8-4f73-8d8a-a97d84e6ad1d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 03:33:44.119028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:44.119028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:44.119028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:44.119028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:44.119028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:44.119028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:44.119028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:44.119028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:44.119028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:44.119028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:44.119028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:44.119028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:33:44.119028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:33:44.119028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:33:44.119028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:33:44.120331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:44.120331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:44.120331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:33:44.120331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:44.120331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:33:44.120331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:44.120331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:44.120331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:44.120331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:44.120331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:44.120331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:44.120331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:44.120331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:44.120331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:44.120331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:44.120331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:44.120331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:44.120331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:44.121262 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:44.121262 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:44.121262 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:33:44.121262 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:33:44.121262 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:33:44.121262 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:33:44.121262 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:33:44.121262 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:33:44.121262 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 03:33:44.121262 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:44.121262 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 390/785] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Oct 17 03:33:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:44.126236 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbcefc7-2b69-4b85-8a13-0312e8ff7538 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:44.231444 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbcefc7-2b69-4b85-8a13-0312e8ff7538 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:44.233807 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:44.237450 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd0f82-1a06-4c5e-9260-b63330a2eb7d None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:44.237743 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd0f82-1a06-4c5e-9260-b63330a2eb7d None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:44.238020 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd0f82-1a06-4c5e-9260-b63330a2eb7d None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:44.239501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 391/786] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:33:44.244423 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db323f0d-3f13-4e26-8a11-65a52e4e7fd9 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:44.244712 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db323f0d-3f13-4e26-8a11-65a52e4e7fd9 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:44.244992 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db323f0d-3f13-4e26-8a11-65a52e4e7fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:44.263660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-db323f0d-3f13-4e26-8a11-65a52e4e7fd9 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:44.265228 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-db323f0d-3f13-4e26-8a11-65a52e4e7fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:44.266364 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6cb56b-6cbc-4878-9104-debeb6663ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:44.266976 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6cb56b-6cbc-4878-9104-debeb6663ce8 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:44.267295 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6cb56b-6cbc-4878-9104-debeb6663ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:44.267527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6cb56b-6cbc-4878-9104-debeb6663ce8 None admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:44.268730 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6cb56b-6cbc-4878-9104-debeb6663ce8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:44.268730 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6cb56b-6cbc-4878-9104-debeb6663ce8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:44.269963 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6cb56b-6cbc-4878-9104-debeb6663ce8 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:44.288942 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 396/787] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Oct 17 03:33:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:44.294633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6cb56b-6cbc-4878-9104-debeb6663ce8 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:44.340983 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 392/788] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:33:44.401418 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6cb56b-6cbc-4878-9104-debeb6663ce8 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:44.435296 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be22d2ca-79e6-486a-99ea-fc3c379a7462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:44.435952 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22d2ca-79e6-486a-99ea-fc3c379a7462 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:44.436237 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22d2ca-79e6-486a-99ea-fc3c379a7462 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:44.436509 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22d2ca-79e6-486a-99ea-fc3c379a7462 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:44.447192 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be22d2ca-79e6-486a-99ea-fc3c379a7462 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:44.448643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be22d2ca-79e6-486a-99ea-fc3c379a7462 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:44.464010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-be22d2ca-79e6-486a-99ea-fc3c379a7462 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:44.464010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:44.464010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:44.464010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:44.464010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:44.464010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:44.464010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:44.464010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:44.464010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:44.464010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:44.464010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:44.464010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:44.464010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:44.464010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:44.464010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:44.464010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:44.464935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:44.464935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:44.464935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:44.464935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:44.464935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:44.464935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:44.464935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:44.464935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:44.464935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:44.464935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:44.464935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:44.464935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:44.464935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:44.464935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:44.464935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:44.464935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:44.464935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:44.465837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:44.465837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:44.465837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:44.466547 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 397/789] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:33:44.471047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db323f0d-3f13-4e26-8a11-65a52e4e7fd9 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:44.548767 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:44.570503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db323f0d-3f13-4e26-8a11-65a52e4e7fd9 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:44.605053 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a0ecd-338d-42bb-b45a-52cf4874b6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:44.605668 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a0ecd-338d-42bb-b45a-52cf4874b6bc None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:44.605935 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a0ecd-338d-42bb-b45a-52cf4874b6bc None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:44.606207 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a0ecd-338d-42bb-b45a-52cf4874b6bc None admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:44.607714 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be22d2ca-79e6-486a-99ea-fc3c379a7462 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:44.608017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a0ecd-338d-42bb-b45a-52cf4874b6bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:44.608353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a0ecd-338d-42bb-b45a-52cf4874b6bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:44.609728 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a0ecd-338d-42bb-b45a-52cf4874b6bc None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:44.626382 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 393/790] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:44.626583 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:44.627400 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a0ecd-338d-42bb-b45a-52cf4874b6bc None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:44.727325 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be22d2ca-79e6-486a-99ea-fc3c379a7462 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:44.738321 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a0ecd-338d-42bb-b45a-52cf4874b6bc None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:44.763853 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b19b8-6f8e-47ae-ae11-d97bc8a649bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:44.764507 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b19b8-6f8e-47ae-ae11-d97bc8a649bf None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:44.764782 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b19b8-6f8e-47ae-ae11-d97bc8a649bf None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:44.765049 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b19b8-6f8e-47ae-ae11-d97bc8a649bf None admin] PATH_INFO: `/v3/services/image` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:44.765785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b19b8-6f8e-47ae-ae11-d97bc8a649bf None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:44.767751 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b19b8-6f8e-47ae-ae11-d97bc8a649bf None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:44.773615 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01921878-32c8-469e-884d-1ddeea9854e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:44.774236 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01921878-32c8-469e-884d-1ddeea9854e5 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:44.774513 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01921878-32c8-469e-884d-1ddeea9854e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:44.774782 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01921878-32c8-469e-884d-1ddeea9854e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:44.776938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-2a8b19b8-6f8e-47ae-ae11-d97bc8a649bf None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 03:33:44.776938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:44.776938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:33:44.776938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:33:44.776938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:33:44.776938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:33:44.776938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:33:44.776938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:44.776938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 03:33:44.776938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:44.776938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:33:44.776938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:44.776938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:44.776938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:44.776938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:44.776938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:44.776938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:44.777867 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:44.777867 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:44.777867 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:44.777867 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:44.777867 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:44.777867 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:44.777867 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:33:44.777867 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:33:44.777867 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:33:44.777867 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:33:44.777867 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:44.777867 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:44.777867 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:44.777867 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:44.777867 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:44.777867 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:44.778692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:44.778692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:44.778692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:44.778692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:44.778692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:44.778692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:44.778692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:44.778692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:44.778692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:44.778692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:44.778692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:33:44.778692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:33:44.778692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 17 03:33:44.778692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:44.783449 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 398/791] 10.222.0.17 () {56 vars in 1105 bytes} [Mon Oct 17 03:33:44 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:33:44.784257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01921878-32c8-469e-884d-1ddeea9854e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:44.784381 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b19b8-6f8e-47ae-ae11-d97bc8a649bf None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:44.785681 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01921878-32c8-469e-884d-1ddeea9854e5 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:44.787684 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:44.798624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-01921878-32c8-469e-884d-1ddeea9854e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:44.798624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:44.798624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:44.798624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:44.798624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:44.798624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:44.798624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:44.798624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:44.798624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:44.798624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:44.798624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:44.798624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:44.798624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:33:44.798624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:33:44.798624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:33:44.802085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:33:44.802085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:44.802085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:44.802085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:44.802085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:44.802085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:44.802085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:44.802085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:44.802085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:44.802085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:44.802085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:44.802085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:44.802085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:44.802085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:44.802085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:44.802085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:44.802085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:44.802951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:33:44.802951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:33:44.802951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:33:44.802951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:33:44.802951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:33:44.802951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:33:44.802951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:44.802951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:44.802951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 394/792] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:33:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:44.805703 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01921878-32c8-469e-884d-1ddeea9854e5 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:44.872318 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:44.897595 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b19b8-6f8e-47ae-ae11-d97bc8a649bf None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:44.915855 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01921878-32c8-469e-884d-1ddeea9854e5 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:44.936298 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c4326-b454-458d-9815-a940628a2d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:44.936831 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c4326-b454-458d-9815-a940628a2d6e None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:44.937110 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c4326-b454-458d-9815-a940628a2d6e None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:44.937377 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c4326-b454-458d-9815-a940628a2d6e None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:44.938001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c4326-b454-458d-9815-a940628a2d6e None admin] RBAC: Adding query filter params (name=image) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:44.938300 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c4326-b454-458d-9815-a940628a2d6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:44.939639 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c4326-b454-458d-9815-a940628a2d6e None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:44.949982 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 399/793] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:44.951340 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c4326-b454-458d-9815-a940628a2d6e None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:44.951695 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ef150-ad65-4441-a796-599da39dbd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:44.952323 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ef150-ad65-4441-a796-599da39dbd39 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:44.952650 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ef150-ad65-4441-a796-599da39dbd39 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:44.952918 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ef150-ad65-4441-a796-599da39dbd39 None admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:44.953628 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ef150-ad65-4441-a796-599da39dbd39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:44.953934 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ef150-ad65-4441-a796-599da39dbd39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:44.955347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ef150-ad65-4441-a796-599da39dbd39 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:44.967695 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 395/794] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Oct 17 03:33:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:33:44.973393 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ef150-ad65-4441-a796-599da39dbd39 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:45.031368 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:45.058972 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c4326-b454-458d-9815-a940628a2d6e None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:45.071972 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ef150-ad65-4441-a796-599da39dbd39 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:45.092269 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038e31eb-fbea-4608-b49a-167cd00a6b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:45.092854 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e31eb-fbea-4608-b49a-167cd00a6b5f None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:45.093129 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e31eb-fbea-4608-b49a-167cd00a6b5f None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:45.093398 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038e31eb-fbea-4608-b49a-167cd00a6b5f None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:45.094010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038e31eb-fbea-4608-b49a-167cd00a6b5f None admin] RBAC: Adding query filter params (type=image) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:45.094313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038e31eb-fbea-4608-b49a-167cd00a6b5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:45.095632 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038e31eb-fbea-4608-b49a-167cd00a6b5f None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:45.105566 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 400/795] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:44 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:33:45.105566 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c8b79-6a62-4930-9fed-ea3ea21a48e5 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:45.105566 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c8b79-6a62-4930-9fed-ea3ea21a48e5 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:45.105566 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5c8b79-6a62-4930-9fed-ea3ea21a48e5 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:45.105566 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 401/796] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:45.106415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c8b79-6a62-4930-9fed-ea3ea21a48e5 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:45.108254 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba430d06-0397-4058-aafd-f108cfb6ba81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:45.108917 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba430d06-0397-4058-aafd-f108cfb6ba81 None admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:45.109197 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba430d06-0397-4058-aafd-f108cfb6ba81 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:45.109473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba430d06-0397-4058-aafd-f108cfb6ba81 None admin] PATH_INFO: `/v3/projects/932cdb9ef7644bf5ba57c6d1d6bce7ff/users/d961b751763a4e49b5d4ca4c795c5bf6/roles/a810fe6483474b65ab4ee8969ce78455` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:45.111631 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba430d06-0397-4058-aafd-f108cfb6ba81 None admin] RBAC: Authorizing `identity:create_grant(project_id=932cdb9ef7644bf5ba57c6d1d6bce7ff, user_id=d961b751763a4e49b5d4ca4c795c5bf6, role_id=a810fe6483474b65ab4ee8969ce78455)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:45.113089 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba430d06-0397-4058-aafd-f108cfb6ba81 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:45.115487 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:45.175630 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 396/797] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Oct 17 03:33:44 2022] PUT /identity/v3/projects/932cdb9ef7644bf5ba57c6d1d6bce7ff/users/d961b751763a4e49b5d4ca4c795c5bf6/roles/a810fe6483474b65ab4ee8969ce78455 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:33:45.176900 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658db915-ea9d-4c26-a5ba-04d3bd67f244 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:45.177188 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658db915-ea9d-4c26-a5ba-04d3bd67f244 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:45.177459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658db915-ea9d-4c26-a5ba-04d3bd67f244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:45.197974 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-658db915-ea9d-4c26-a5ba-04d3bd67f244 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:45.199108 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-658db915-ea9d-4c26-a5ba-04d3bd67f244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:45.210349 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5c8b79-6a62-4930-9fed-ea3ea21a48e5 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:45.251961 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0afd1c0-ac24-4511-ae1a-26e5880a559c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:45.252594 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0afd1c0-ac24-4511-ae1a-26e5880a559c None admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:45.252878 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0afd1c0-ac24-4511-ae1a-26e5880a559c None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:45.253186 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0afd1c0-ac24-4511-ae1a-26e5880a559c None admin] PATH_INFO: `/v3/endpoints` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:45.253873 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0afd1c0-ac24-4511-ae1a-26e5880a559c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:45.255266 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0afd1c0-ac24-4511-ae1a-26e5880a559c None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:45.279656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 397/798] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:45 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:33:45.349589 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 402/799] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:33:45 2022] POST /identity/v3/endpoints => generated 327 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:33:45.556558 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658db915-ea9d-4c26-a5ba-04d3bd67f244 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:45.606956 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:45.658826 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658db915-ea9d-4c26-a5ba-04d3bd67f244 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:45.706021 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e38ad7-f289-407e-9215-fd9300ea0fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:45.706662 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e38ad7-f289-407e-9215-fd9300ea0fa3 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:45.706940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e38ad7-f289-407e-9215-fd9300ea0fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:45.707209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e38ad7-f289-407e-9215-fd9300ea0fa3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:45.708326 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e38ad7-f289-407e-9215-fd9300ea0fa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:45.710269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e38ad7-f289-407e-9215-fd9300ea0fa3 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:45.715576 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 398/800] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:45.718262 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0afd1c0-ac24-4511-ae1a-26e5880a559c None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:45.781623 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:45.820862 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0afd1c0-ac24-4511-ae1a-26e5880a559c None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:45.854773 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e2f27-10cb-4db9-9bd9-ae0ff1cad623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:45.855558 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e2f27-10cb-4db9-9bd9-ae0ff1cad623 None admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:45.856317 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e2f27-10cb-4db9-9bd9-ae0ff1cad623 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:45.856607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e2f27-10cb-4db9-9bd9-ae0ff1cad623 None admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:45.857265 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e2f27-10cb-4db9-9bd9-ae0ff1cad623 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:45.858710 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e2f27-10cb-4db9-9bd9-ae0ff1cad623 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:45.949495 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 403/801] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Oct 17 03:33:45 2022] POST /identity/v3/projects => generated 311 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:33:47.331833 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9d536-8755-431b-b545-81990180e139 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:47.332355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9d536-8755-431b-b545-81990180e139 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:47.332355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9d536-8755-431b-b545-81990180e139 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:47.333756 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 399/802] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:47.339694 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867da07-6cea-4e8c-93b2-9130b0e0475b None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:47.340119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867da07-6cea-4e8c-93b2-9130b0e0475b None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:47.340296 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867da07-6cea-4e8c-93b2-9130b0e0475b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:47.358477 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-8867da07-6cea-4e8c-93b2-9130b0e0475b None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:47.359648 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-8867da07-6cea-4e8c-93b2-9130b0e0475b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:47.431345 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 404/803] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:33:47.501110 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c84d7a-ecac-4de5-9130-1e55587cbc65 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:47.501431 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c84d7a-ecac-4de5-9130-1e55587cbc65 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:47.501659 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c84d7a-ecac-4de5-9130-1e55587cbc65 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:47.503133 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 400/804] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:47.508619 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38dd016-6e0e-4312-8eec-a2579aa153e9 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:47.508912 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38dd016-6e0e-4312-8eec-a2579aa153e9 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:47.509188 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38dd016-6e0e-4312-8eec-a2579aa153e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:47.527574 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-d38dd016-6e0e-4312-8eec-a2579aa153e9 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:47.528759 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-d38dd016-6e0e-4312-8eec-a2579aa153e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:47.567643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:47.596036 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 405/805] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:33:47.701300 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c84d7a-ecac-4de5-9130-1e55587cbc65 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:47.806416 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c84d7a-ecac-4de5-9130-1e55587cbc65 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:47.821121 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:47.842801 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea06b11-ccda-4f6f-83cd-25db1f545b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:47.843436 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea06b11-ccda-4f6f-83cd-25db1f545b73 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:47.843769 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea06b11-ccda-4f6f-83cd-25db1f545b73 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:47.844051 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea06b11-ccda-4f6f-83cd-25db1f545b73 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:47.854852 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea06b11-ccda-4f6f-83cd-25db1f545b73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:33:47.859226 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38dd016-6e0e-4312-8eec-a2579aa153e9 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:47.860578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-3ea06b11-ccda-4f6f-83cd-25db1f545b73 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 03:33:47.860578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:47.860578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:47.860578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:47.860578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:47.860578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:47.860578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:47.860578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:47.860578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:47.860578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:47.860578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:47.860578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:47.860578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:33:47.860578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:33:47.860578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:33:47.860578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:33:47.861524 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:33:47.861524 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:33:47.861524 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:33:47.861524 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:33:47.861524 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:47.861524 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:47.861524 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:47.861524 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:47.861524 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:47.861524 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:47.861524 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:47.861524 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:47.861524 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:47.861524 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:47.861524 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:47.861524 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:47.864110 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:47.864110 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:47.864110 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:47.864110 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:47.864110 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:33:47.864110 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:33:47.864110 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:33:47.864110 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:33:47.864110 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:33:47.864110 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:33:47.864110 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 17 03:33:47.864110 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:47.864110 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 401/806] 10.222.0.17 () {56 vars in 1120 bytes} [Mon Oct 17 03:33:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:47.868439 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea06b11-ccda-4f6f-83cd-25db1f545b73 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:47.965296 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38dd016-6e0e-4312-8eec-a2579aa153e9 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:47.973266 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea06b11-ccda-4f6f-83cd-25db1f545b73 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:48.005452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02021f3-8d01-4e1b-aa5a-488694a1ec8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:48.006077 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02021f3-8d01-4e1b-aa5a-488694a1ec8f None admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:48.006960 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02021f3-8d01-4e1b-aa5a-488694a1ec8f None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:48.007157 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02021f3-8d01-4e1b-aa5a-488694a1ec8f None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:48.007880 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02021f3-8d01-4e1b-aa5a-488694a1ec8f None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:48.008992 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30f111-13b2-4d8c-82be-dd1404ce5653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:48.009269 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02021f3-8d01-4e1b-aa5a-488694a1ec8f None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:48.009627 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30f111-13b2-4d8c-82be-dd1404ce5653 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:48.009907 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30f111-13b2-4d8c-82be-dd1404ce5653 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:48.010179 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30f111-13b2-4d8c-82be-dd1404ce5653 None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:48.010863 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e30f111-13b2-4d8c-82be-dd1404ce5653 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:48.011166 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e30f111-13b2-4d8c-82be-dd1404ce5653 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:48.012582 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e30f111-13b2-4d8c-82be-dd1404ce5653 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:48.023326 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 402/807] 10.222.0.17 () {56 vars in 1115 bytes} [Mon Oct 17 03:33:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:48.030172 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30f111-13b2-4d8c-82be-dd1404ce5653 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:48.048134 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.sql.core [None req-c02021f3-8d01-4e1b-aa5a-488694a1ec8f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 03:33:48.048134 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 03:33:48.048134 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [parameters: {'id': '2b9a450eefde443a90b8527bcf87dc53', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 17 03:33:48.048134 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88300) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 17 03:33:48.055647 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-c02021f3-8d01-4e1b-aa5a-488694a1ec8f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 17 03:33:48.055647 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:48.055647 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 03:33:48.055647 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 03:33:48.055647 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 03:33:48.055647 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 03:33:48.055647 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 03:33:48.055647 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 03:33:48.055647 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 03:33:48.055647 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application conn.query(q) Oct 17 03:33:48.055647 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 03:33:48.055647 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 03:33:48.055647 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 03:33:48.055647 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application result.read() Oct 17 03:33:48.056594 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 03:33:48.056594 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 03:33:48.056594 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 03:33:48.056594 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 03:33:48.056594 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 03:33:48.056594 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 03:33:48.056594 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 03:33:48.056594 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 03:33:48.056594 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 03:33:48.056594 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:48.056594 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 17 03:33:48.056594 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:48.056594 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:48.056594 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 17 03:33:48.056594 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 17 03:33:48.056594 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 17 03:33:48.056594 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 17 03:33:48.056594 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 03:33:48.057533 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application next(self.gen) Oct 17 03:33:48.057533 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 17 03:33:48.057533 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application yield resource Oct 17 03:33:48.057533 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 17 03:33:48.057533 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application next(self.gen) Oct 17 03:33:48.057533 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 17 03:33:48.057533 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self.session.rollback() Oct 17 03:33:48.057533 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 17 03:33:48.057533 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self.force_reraise() Oct 17 03:33:48.057533 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 17 03:33:48.057533 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise self.value Oct 17 03:33:48.057533 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 17 03:33:48.057533 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 17 03:33:48.057533 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 17 03:33:48.057533 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application session.commit() Oct 17 03:33:48.057533 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 17 03:33:48.057533 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 17 03:33:48.058384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 17 03:33:48.058384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._prepare_impl() Oct 17 03:33:48.058384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 17 03:33:48.058384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self.session.flush() Oct 17 03:33:48.058384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 17 03:33:48.058384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._flush(objects) Oct 17 03:33:48.058384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 17 03:33:48.058384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 17 03:33:48.058384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 17 03:33:48.058384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application compat.raise_( Oct 17 03:33:48.058384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 03:33:48.058384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception Oct 17 03:33:48.058384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 17 03:33:48.058384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application flush_context.execute() Oct 17 03:33:48.058384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 17 03:33:48.058384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rec.execute(self) Oct 17 03:33:48.058384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 17 03:33:48.059231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 17 03:33:48.059231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 17 03:33:48.059231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 17 03:33:48.059231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 17 03:33:48.059231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 17 03:33:48.059231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 17 03:33:48.059231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 17 03:33:48.059231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 17 03:33:48.059231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 17 03:33:48.059231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 17 03:33:48.059231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 17 03:33:48.059231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 17 03:33:48.059231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 17 03:33:48.059231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 17 03:33:48.059231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 17 03:33:48.060030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 17 03:33:48.060030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception Oct 17 03:33:48.060030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 17 03:33:48.060030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 17 03:33:48.060030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 17 03:33:48.060030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 17 03:33:48.060030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 17 03:33:48.060030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application result = self._query(query) Oct 17 03:33:48.060030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 17 03:33:48.060030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application conn.query(q) Oct 17 03:33:48.060030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 17 03:33:48.060030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 17 03:33:48.060030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 17 03:33:48.060030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application result.read() Oct 17 03:33:48.060030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 17 03:33:48.060030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 17 03:33:48.060030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 17 03:33:48.060864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 17 03:33:48.060864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 17 03:33:48.060864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 17 03:33:48.060864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 17 03:33:48.060864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 17 03:33:48.060864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 17 03:33:48.060864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 17 03:33:48.060864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [parameters: {'id': '2b9a450eefde443a90b8527bcf87dc53', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 17 03:33:48.060864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 17 03:33:48.060864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:48.060864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:33:48.060864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:48.060864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:48.060864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 17 03:33:48.060864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 17 03:33:48.061665 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 17 03:33:48.061665 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 17 03:33:48.061665 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 17 03:33:48.061665 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:48.061665 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:33:48.061665 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:48.061665 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:48.061665 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:48.061665 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:48.061665 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:48.061665 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:48.061665 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:48.061665 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:48.061665 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:48.061665 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:48.061665 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:48.061665 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:48.062543 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 17 03:33:48.062543 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 17 03:33:48.062543 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:48.062543 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:48.062543 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 17 03:33:48.062543 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 17 03:33:48.062543 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:48.062543 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:48.062543 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 17 03:33:48.062543 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 17 03:33:48.062543 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 17 03:33:48.062543 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:48.062543 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 406/808] 10.222.0.17 () {60 vars in 1137 bytes} [Mon Oct 17 03:33:47 2022] POST /identity/v3/domains => generated 190 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:33:48.063217 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02021f3-8d01-4e1b-aa5a-488694a1ec8f None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:48.075626 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:48.106549 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:48.145042 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30f111-13b2-4d8c-82be-dd1404ce5653 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:48.169273 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02021f3-8d01-4e1b-aa5a-488694a1ec8f None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:48.179042 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d0cb5-79b3-454d-bff4-67e0d8a392f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:48.179669 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d0cb5-79b3-454d-bff4-67e0d8a392f7 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:48.179980 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d0cb5-79b3-454d-bff4-67e0d8a392f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:48.180256 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d0cb5-79b3-454d-bff4-67e0d8a392f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:48.190058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d0cb5-79b3-454d-bff4-67e0d8a392f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:48.191485 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d0cb5-79b3-454d-bff4-67e0d8a392f7 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:48.203392 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512e4275-07b9-4252-9940-e879c1cfe8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:48.204050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e4275-07b9-4252-9940-e879c1cfe8d5 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:48.204333 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e4275-07b9-4252-9940-e879c1cfe8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:48.204602 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e4275-07b9-4252-9940-e879c1cfe8d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:48.206038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-1e4d0cb5-79b3-454d-bff4-67e0d8a392f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:48.206038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:48.206038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:48.206038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:48.206038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:48.206038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:48.206038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:48.206038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:48.206038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:48.206038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:48.206038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:48.206038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:48.206038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:48.206038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:48.206038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:48.206038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:48.206939 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:48.206939 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:48.206939 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:48.206939 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:48.206939 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:48.206939 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:48.206939 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:48.206939 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:48.206939 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:48.206939 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:48.206939 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:48.206939 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:48.206939 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:48.206939 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:48.206939 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:48.206939 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:48.206939 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:48.207993 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:48.207993 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:48.207993 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:48.207993 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 403/809] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:33:48.208278 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b20a05-fb68-4121-bfe8-1f37c716e918 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:48.208558 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b20a05-fb68-4121-bfe8-1f37c716e918 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:48.208828 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b20a05-fb68-4121-bfe8-1f37c716e918 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:48.210786 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 404/810] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:48.214928 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b20a05-fb68-4121-bfe8-1f37c716e918 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:48.219061 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512e4275-07b9-4252-9940-e879c1cfe8d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:48.220513 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512e4275-07b9-4252-9940-e879c1cfe8d5 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:48.234886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-512e4275-07b9-4252-9940-e879c1cfe8d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:48.234886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:48.234886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:48.234886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:48.234886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:48.234886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:48.234886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:48.234886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:48.234886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:48.234886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:48.234886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:48.234886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:48.234886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:48.234886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:48.234886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:48.234886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:48.236057 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:48.236057 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:48.236057 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:48.236057 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:48.236057 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:48.236057 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:48.236057 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:48.236057 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:48.236057 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:48.236057 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:48.236057 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:48.236057 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:48.236057 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:48.236057 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:48.236057 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:48.236057 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:48.236057 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:48.236956 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:48.236956 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:48.236956 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:48.236956 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 407/811] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:48.237416 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bef6ae-5f8d-4515-9778-7605f7a45414 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:48.237696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bef6ae-5f8d-4515-9778-7605f7a45414 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:48.237970 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bef6ae-5f8d-4515-9778-7605f7a45414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:48.255167 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-d6bef6ae-5f8d-4515-9778-7605f7a45414 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:48.256881 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-d6bef6ae-5f8d-4515-9778-7605f7a45414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:48.303903 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:48.319584 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b20a05-fb68-4121-bfe8-1f37c716e918 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:48.322602 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 408/812] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:33:48.323431 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bef6ae-5f8d-4515-9778-7605f7a45414 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:48.353049 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:48.356156 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd88d9-5890-4366-bd9b-27f0c5c88c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:48.356837 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd88d9-5890-4366-bd9b-27f0c5c88c94 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:48.357063 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd88d9-5890-4366-bd9b-27f0c5c88c94 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:48.357341 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd88d9-5890-4366-bd9b-27f0c5c88c94 None admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:48.357990 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd88d9-5890-4366-bd9b-27f0c5c88c94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:48.358300 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd88d9-5890-4366-bd9b-27f0c5c88c94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:48.359659 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd88d9-5890-4366-bd9b-27f0c5c88c94 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:48.371981 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 405/813] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:48.377631 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd88d9-5890-4366-bd9b-27f0c5c88c94 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:48.436963 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bef6ae-5f8d-4515-9778-7605f7a45414 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:48.473910 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83d0e3-0db1-42d0-88d2-9b292faaa7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:48.474635 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83d0e3-0db1-42d0-88d2-9b292faaa7b9 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:48.475039 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83d0e3-0db1-42d0-88d2-9b292faaa7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:48.475333 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83d0e3-0db1-42d0-88d2-9b292faaa7b9 None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:48.476002 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83d0e3-0db1-42d0-88d2-9b292faaa7b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:48.476308 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83d0e3-0db1-42d0-88d2-9b292faaa7b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:48.477676 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83d0e3-0db1-42d0-88d2-9b292faaa7b9 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:48.478361 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd88d9-5890-4366-bd9b-27f0c5c88c94 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:48.496178 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 409/814] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:48.514215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112bc75-57f1-4a69-89fb-072aa4b5d1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:48.514959 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112bc75-57f1-4a69-89fb-072aa4b5d1b2 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:48.515301 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112bc75-57f1-4a69-89fb-072aa4b5d1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:48.515631 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6112bc75-57f1-4a69-89fb-072aa4b5d1b2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:48.524497 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6112bc75-57f1-4a69-89fb-072aa4b5d1b2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:48.526057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6112bc75-57f1-4a69-89fb-072aa4b5d1b2 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:48.538570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-6112bc75-57f1-4a69-89fb-072aa4b5d1b2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 17 03:33:48.538570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:48.538570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:48.538570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:48.538570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:48.538570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:48.538570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:48.538570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:48.538570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:48.538570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:48.538570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:48.538570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:48.538570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:33:48.538570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:33:48.538570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:33:48.538570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:33:48.539791 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:48.539791 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:48.539791 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:33:48.539791 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:48.539791 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:33:48.539791 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:33:48.539791 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:48.539791 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:48.539791 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:48.539791 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:48.539791 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:48.539791 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:48.539791 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:48.539791 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:48.539791 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:48.539791 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:48.539791 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:48.539791 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:48.540967 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:48.540967 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:48.540967 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:33:48.540967 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:33:48.540967 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:33:48.540967 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:33:48.540967 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:33:48.540967 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:33:48.540967 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 17 03:33:48.540967 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:48.541779 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 406/815] 10.222.0.17 () {56 vars in 1093 bytes} [Mon Oct 17 03:33:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:33:48.547319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83d0e3-0db1-42d0-88d2-9b292faaa7b9 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:48.595846 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112bc75-57f1-4a69-89fb-072aa4b5d1b2 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:48.606510 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:48.721149 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6112bc75-57f1-4a69-89fb-072aa4b5d1b2 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:48.757957 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:48.761781 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d5dd8-c35b-467c-af53-dd02660ad632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:48.762000 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d5dd8-c35b-467c-af53-dd02660ad632 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:48.762271 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d5dd8-c35b-467c-af53-dd02660ad632 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:48.763461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d5dd8-c35b-467c-af53-dd02660ad632 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:48.763928 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d5dd8-c35b-467c-af53-dd02660ad632 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:48.765118 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d5dd8-c35b-467c-af53-dd02660ad632 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:48.767019 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 407/816] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:33:48.773459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d5dd8-c35b-467c-af53-dd02660ad632 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:48.785212 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83d0e3-0db1-42d0-88d2-9b292faaa7b9 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:48.822553 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49d600-e37c-417c-9cf2-f63da7650180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:48.823158 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49d600-e37c-417c-9cf2-f63da7650180 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:48.823433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49d600-e37c-417c-9cf2-f63da7650180 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:48.823696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49d600-e37c-417c-9cf2-f63da7650180 None admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:48.824439 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49d600-e37c-417c-9cf2-f63da7650180 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:48.824738 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49d600-e37c-417c-9cf2-f63da7650180 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:48.826133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49d600-e37c-417c-9cf2-f63da7650180 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:48.843262 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 410/817] 10.222.0.17 () {56 vars in 1133 bytes} [Mon Oct 17 03:33:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:48.848921 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49d600-e37c-417c-9cf2-f63da7650180 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:48.855683 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:48.879462 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d5dd8-c35b-467c-af53-dd02660ad632 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:48.913763 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc3e1b-0235-497a-abfd-5741c68751b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:48.914367 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc3e1b-0235-497a-abfd-5741c68751b0 None admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:48.914644 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc3e1b-0235-497a-abfd-5741c68751b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:48.914916 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc3e1b-0235-497a-abfd-5741c68751b0 None admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:48.915558 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debc3e1b-0235-497a-abfd-5741c68751b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:48.916998 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debc3e1b-0235-497a-abfd-5741c68751b0 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:48.945092 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49d600-e37c-417c-9cf2-f63da7650180 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:48.977395 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570b7e2-f20f-4ffb-8c04-c390e1d7cdd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:48.977992 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570b7e2-f20f-4ffb-8c04-c390e1d7cdd5 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:48.978269 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570b7e2-f20f-4ffb-8c04-c390e1d7cdd5 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:48.978538 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7570b7e2-f20f-4ffb-8c04-c390e1d7cdd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:48.986524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:48.998581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 408/818] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Oct 17 03:33:48 2022] POST /identity/v3/users => generated 285 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:33:49.048332 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570b7e2-f20f-4ffb-8c04-c390e1d7cdd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:49.049886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7570b7e2-f20f-4ffb-8c04-c390e1d7cdd5 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:49.153431 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 411/819] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:33:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 17 03:33:49.159401 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc3e1b-0235-497a-abfd-5741c68751b0 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:49.260743 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc3e1b-0235-497a-abfd-5741c68751b0 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:49.287539 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:49.292813 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40657f5-fe90-4430-a432-5c41a85dfbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:49.293434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40657f5-fe90-4430-a432-5c41a85dfbae None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:49.293711 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40657f5-fe90-4430-a432-5c41a85dfbae None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:49.293980 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40657f5-fe90-4430-a432-5c41a85dfbae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:49.304700 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40657f5-fe90-4430-a432-5c41a85dfbae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:49.306120 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40657f5-fe90-4430-a432-5c41a85dfbae None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:49.320253 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-a40657f5-fe90-4430-a432-5c41a85dfbae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:49.320253 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:49.320253 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:49.320253 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:49.320253 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:49.320253 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:49.320253 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:49.320253 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:49.320253 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:49.320253 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:49.320253 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:49.320253 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:49.320253 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:33:49.320253 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:33:49.320253 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:33:49.320253 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:33:49.321231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:49.321231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:49.321231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:49.321231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:49.321231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:49.321231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:49.321231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:49.321231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:49.321231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:49.321231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:49.321231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:49.321231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:49.321231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:49.321231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:49.321231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:49.321231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:49.321231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:33:49.322117 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:33:49.322117 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:33:49.322117 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:49.322117 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 409/820] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:49.327001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570b7e2-f20f-4ffb-8c04-c390e1d7cdd5 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:49.364599 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:49.421766 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7570b7e2-f20f-4ffb-8c04-c390e1d7cdd5 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:49.454289 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc0775-365f-4347-a3fa-760d4b686ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:49.454882 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc0775-365f-4347-a3fa-760d4b686ea7 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:49.455159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc0775-365f-4347-a3fa-760d4b686ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:49.455424 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc0775-365f-4347-a3fa-760d4b686ea7 None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:49.456080 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc0775-365f-4347-a3fa-760d4b686ea7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:49.456377 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc0775-365f-4347-a3fa-760d4b686ea7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:49.457710 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc0775-365f-4347-a3fa-760d4b686ea7 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:49.468687 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 412/821] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:33:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:49.474246 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40657f5-fe90-4430-a432-5c41a85dfbae None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:49.573385 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40657f5-fe90-4430-a432-5c41a85dfbae None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:49.606528 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade09f5-86f7-4123-b88b-7273c3bb52dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:49.607160 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade09f5-86f7-4123-b88b-7273c3bb52dc None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:49.607442 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade09f5-86f7-4123-b88b-7273c3bb52dc None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:49.607714 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade09f5-86f7-4123-b88b-7273c3bb52dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:49.614162 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade09f5-86f7-4123-b88b-7273c3bb52dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:49.616239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade09f5-86f7-4123-b88b-7273c3bb52dc None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:49.627515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-aade09f5-86f7-4123-b88b-7273c3bb52dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:49.627515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:49.627515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:49.627515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:49.627515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:49.627515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:49.627515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:49.627515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:49.627515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:49.627515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:49.627515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:49.627515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:49.627515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:33:49.627515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:33:49.627515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:33:49.628470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:33:49.628470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:49.628470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:49.628470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:49.628470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:49.628470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:49.628470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:49.628470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:49.628470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:49.628470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:49.628470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:49.628470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:49.628470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:49.628470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:49.628470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:49.628470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:49.628470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:49.629348 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:33:49.629348 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:33:49.629348 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:33:49.629348 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:33:49.629348 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:33:49.629348 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:33:49.629348 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:33:49.629348 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:49.629348 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 410/822] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:33:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:49.634797 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc0775-365f-4347-a3fa-760d4b686ea7 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:49.730720 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:49.734494 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc0775-365f-4347-a3fa-760d4b686ea7 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:49.767343 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63fe0e-c07e-40f7-aca4-5cd63388500c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:49.767959 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63fe0e-c07e-40f7-aca4-5cd63388500c None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:49.768238 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63fe0e-c07e-40f7-aca4-5cd63388500c None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:49.768505 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63fe0e-c07e-40f7-aca4-5cd63388500c None admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:49.769183 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63fe0e-c07e-40f7-aca4-5cd63388500c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:49.769481 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63fe0e-c07e-40f7-aca4-5cd63388500c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:49.770864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63fe0e-c07e-40f7-aca4-5cd63388500c None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:49.782313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 413/823] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Oct 17 03:33:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:49.788846 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade09f5-86f7-4123-b88b-7273c3bb52dc None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:49.831725 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:49.886927 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade09f5-86f7-4123-b88b-7273c3bb52dc None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:49.922716 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22142d42-48af-4a2b-b894-f4f760386e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:49.923323 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22142d42-48af-4a2b-b894-f4f760386e37 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:49.923606 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22142d42-48af-4a2b-b894-f4f760386e37 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:49.923918 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22142d42-48af-4a2b-b894-f4f760386e37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:49.924648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22142d42-48af-4a2b-b894-f4f760386e37 None admin] RBAC: Adding query filter params (role.id=a810fe6483474b65ab4ee8969ce78455, scope.project.id=932cdb9ef7644bf5ba57c6d1d6bce7ff, user.id=d961b751763a4e49b5d4ca4c795c5bf6) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:49.924959 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22142d42-48af-4a2b-b894-f4f760386e37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:49.926408 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22142d42-48af-4a2b-b894-f4f760386e37 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:49.932112 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 411/824] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Oct 17 03:33:49 2022] GET /identity/v3/role_assignments?user.id=d961b751763a4e49b5d4ca4c795c5bf6&scope.project.id=932cdb9ef7644bf5ba57c6d1d6bce7ff&role.id=a810fe6483474b65ab4ee8969ce78455 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:33:51.157879 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faf3c36-e6bf-47ac-a6cf-acf34c8af301 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:51.158439 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faf3c36-e6bf-47ac-a6cf-acf34c8af301 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:51.158439 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faf3c36-e6bf-47ac-a6cf-acf34c8af301 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:51.159749 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 414/825] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:51.165256 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1686cca-6aac-4fd1-9027-f3cb4254cb9d None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:51.165657 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1686cca-6aac-4fd1-9027-f3cb4254cb9d None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:51.165805 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1686cca-6aac-4fd1-9027-f3cb4254cb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:51.184025 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-c1686cca-6aac-4fd1-9027-f3cb4254cb9d None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:51.185196 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-c1686cca-6aac-4fd1-9027-f3cb4254cb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:51.249578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 412/826] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:33:51.512472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faf3c36-e6bf-47ac-a6cf-acf34c8af301 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:51.609901 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faf3c36-e6bf-47ac-a6cf-acf34c8af301 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:51.641492 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86159532-3a11-4de2-a6a0-72fd53c7e876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:51.642098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86159532-3a11-4de2-a6a0-72fd53c7e876 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:51.642374 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86159532-3a11-4de2-a6a0-72fd53c7e876 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:51.642642 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86159532-3a11-4de2-a6a0-72fd53c7e876 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:51.652998 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86159532-3a11-4de2-a6a0-72fd53c7e876 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:33:51.657790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-86159532-3a11-4de2-a6a0-72fd53c7e876 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:33:51.657790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:51.657790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:51.657790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:51.657790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:51.657790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:51.657790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:51.657790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:51.657790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:51.657790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:51.657790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:51.657790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:51.657790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:33:51.657790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:33:51.657790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:33:51.657790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:33:51.660176 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:33:51.660176 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:33:51.660176 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:33:51.660176 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:33:51.660176 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:51.660176 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:51.660176 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:51.660176 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:51.660176 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:51.660176 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:51.660176 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:51.660176 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:51.660176 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:51.660176 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:51.660176 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:51.660176 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:51.661018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:51.661018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:51.661018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:51.661018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:51.661018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:33:51.661018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:33:51.661018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:33:51.661018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:33:51.661018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:33:51.661018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:33:51.661018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:33:51.661018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:33:51.661018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 415/827] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Oct 17 03:33:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:51.665426 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1686cca-6aac-4fd1-9027-f3cb4254cb9d None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:51.698098 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:51.760948 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1686cca-6aac-4fd1-9027-f3cb4254cb9d None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:51.793309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ec072-37df-4b06-af54-654ce9ded514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:51.793846 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ec072-37df-4b06-af54-654ce9ded514 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:51.794121 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ec072-37df-4b06-af54-654ce9ded514 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:51.794390 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ec072-37df-4b06-af54-654ce9ded514 None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:51.795041 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ec072-37df-4b06-af54-654ce9ded514 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:51.795339 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ec072-37df-4b06-af54-654ce9ded514 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:51.796709 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ec072-37df-4b06-af54-654ce9ded514 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:51.804313 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 413/828] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:33:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:51.810063 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86159532-3a11-4de2-a6a0-72fd53c7e876 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:51.829089 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:51.905193 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86159532-3a11-4de2-a6a0-72fd53c7e876 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:51.936271 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6eebe-0801-44d3-9900-e7f0c717c68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:51.936875 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6eebe-0801-44d3-9900-e7f0c717c68c None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:51.937149 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6eebe-0801-44d3-9900-e7f0c717c68c None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:51.937413 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c6eebe-0801-44d3-9900-e7f0c717c68c None admin] PATH_INFO: `/v3/users/ffc48e6f1c1449d39ef5649faf585bd1` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:51.950139 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c6eebe-0801-44d3-9900-e7f0c717c68c None admin] RBAC: Authorizing `identity:get_user(user_id=ffc48e6f1c1449d39ef5649faf585bd1)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:51.951533 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c6eebe-0801-44d3-9900-e7f0c717c68c None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:51.953089 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 416/829] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:33:51 2022] GET /identity/v3/users/ffc48e6f1c1449d39ef5649faf585bd1 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:51.958726 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ec072-37df-4b06-af54-654ce9ded514 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:52.050864 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:52.054403 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ec072-37df-4b06-af54-654ce9ded514 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:52.087578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1811ec9-28d2-45ce-83c3-9f4a891190b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:52.088456 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1811ec9-28d2-45ce-83c3-9f4a891190b3 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:52.088742 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1811ec9-28d2-45ce-83c3-9f4a891190b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:52.089012 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1811ec9-28d2-45ce-83c3-9f4a891190b3 None admin] PATH_INFO: `/v3/projects/4b2d78dfa47b4129a0cd63d58a3c4554` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:52.090107 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1811ec9-28d2-45ce-83c3-9f4a891190b3 None admin] RBAC: Authorizing `identity:get_project(project_id=4b2d78dfa47b4129a0cd63d58a3c4554)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:52.091501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1811ec9-28d2-45ce-83c3-9f4a891190b3 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:52.093104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 414/830] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:33:51 2022] GET /identity/v3/projects/4b2d78dfa47b4129a0cd63d58a3c4554 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:52.098787 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6eebe-0801-44d3-9900-e7f0c717c68c None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:52.172200 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:52.194879 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c6eebe-0801-44d3-9900-e7f0c717c68c None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:52.227170 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717dd0ba-7f8d-48be-a6dc-06adb1720849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:52.227839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717dd0ba-7f8d-48be-a6dc-06adb1720849 None admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:52.228119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717dd0ba-7f8d-48be-a6dc-06adb1720849 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:52.228389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717dd0ba-7f8d-48be-a6dc-06adb1720849 None admin] PATH_INFO: `/v3/projects/4b2d78dfa47b4129a0cd63d58a3c4554/users/ffc48e6f1c1449d39ef5649faf585bd1/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:52.230252 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717dd0ba-7f8d-48be-a6dc-06adb1720849 None admin] RBAC: Authorizing `identity:create_grant(project_id=4b2d78dfa47b4129a0cd63d58a3c4554, user_id=ffc48e6f1c1449d39ef5649faf585bd1, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:52.231641 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717dd0ba-7f8d-48be-a6dc-06adb1720849 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:52.298560 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 417/831] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Oct 17 03:33:52 2022] PUT /identity/v3/projects/4b2d78dfa47b4129a0cd63d58a3c4554/users/ffc48e6f1c1449d39ef5649faf585bd1/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:33:54.406596 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866cf53c-36c3-4100-a6bb-b4764ad7152b None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:54.407173 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866cf53c-36c3-4100-a6bb-b4764ad7152b None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:54.407173 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866cf53c-36c3-4100-a6bb-b4764ad7152b None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:54.408512 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 415/832] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:54.413839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4a8a7-6b1b-45ea-9b53-e5e2995229b3 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:54.414276 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4a8a7-6b1b-45ea-9b53-e5e2995229b3 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:54.414405 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4a8a7-6b1b-45ea-9b53-e5e2995229b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:54.432529 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-f0e4a8a7-6b1b-45ea-9b53-e5e2995229b3 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:54.433706 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-f0e4a8a7-6b1b-45ea-9b53-e5e2995229b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:54.497145 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 418/833] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:33:54.765531 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866cf53c-36c3-4100-a6bb-b4764ad7152b None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:54.864808 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866cf53c-36c3-4100-a6bb-b4764ad7152b None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:54.897798 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06106fee-455c-4bb7-a65d-11dc7701ddec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:54.898441 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06106fee-455c-4bb7-a65d-11dc7701ddec None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:54.898721 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06106fee-455c-4bb7-a65d-11dc7701ddec None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:54.898996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06106fee-455c-4bb7-a65d-11dc7701ddec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:54.908874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06106fee-455c-4bb7-a65d-11dc7701ddec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:33:54.914211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-06106fee-455c-4bb7-a65d-11dc7701ddec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:33:54.914211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:33:54.914211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:33:54.914211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:33:54.914211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:33:54.914211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:33:54.914211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:33:54.914211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:33:54.914211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:33:54.914211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:33:54.914211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:33:54.914211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:33:54.914211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:33:54.914211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:33:54.914211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:33:54.914211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:33:54.915355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:33:54.915355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:33:54.915355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:33:54.915355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:33:54.915355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:33:54.915355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:33:54.915355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:33:54.915355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:33:54.915355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:33:54.915355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:33:54.915355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:33:54.915355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:33:54.915355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:33:54.915355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:33:54.915355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:33:54.915355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:33:54.916412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:33:54.916412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:33:54.916412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:33:54.916412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:33:54.916412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:33:54.916412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:33:54.916412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:33:54.916412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:33:54.916412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:33:54.916412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:33:54.916412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:33:54.916412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:33:54.916412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 416/834] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Oct 17 03:33:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:33:54.921276 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e4a8a7-6b1b-45ea-9b53-e5e2995229b3 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:54.983838 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:55.016430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e4a8a7-6b1b-45ea-9b53-e5e2995229b3 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:55.048705 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac1e21-0f76-4965-9800-8b0bae289f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:55.049289 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac1e21-0f76-4965-9800-8b0bae289f21 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:55.049565 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac1e21-0f76-4965-9800-8b0bae289f21 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:55.049834 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac1e21-0f76-4965-9800-8b0bae289f21 None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:55.050481 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac1e21-0f76-4965-9800-8b0bae289f21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:55.050778 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac1e21-0f76-4965-9800-8b0bae289f21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:55.052137 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac1e21-0f76-4965-9800-8b0bae289f21 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:55.060375 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 419/835] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:33:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:55.066051 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06106fee-455c-4bb7-a65d-11dc7701ddec None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:55.093740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:55.164934 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06106fee-455c-4bb7-a65d-11dc7701ddec None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:55.197099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566e8c1-a8a9-48d0-ae3b-8be070ae1942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:55.197711 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566e8c1-a8a9-48d0-ae3b-8be070ae1942 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:55.197990 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566e8c1-a8a9-48d0-ae3b-8be070ae1942 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:55.198262 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566e8c1-a8a9-48d0-ae3b-8be070ae1942 None admin] PATH_INFO: `/v3/users/ffc48e6f1c1449d39ef5649faf585bd1` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:55.199361 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1566e8c1-a8a9-48d0-ae3b-8be070ae1942 None admin] RBAC: Authorizing `identity:get_user(user_id=ffc48e6f1c1449d39ef5649faf585bd1)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:55.200816 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1566e8c1-a8a9-48d0-ae3b-8be070ae1942 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:55.202354 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 417/836] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:33:55 2022] GET /identity/v3/users/ffc48e6f1c1449d39ef5649faf585bd1 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:55.207955 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac1e21-0f76-4965-9800-8b0bae289f21 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:55.302539 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac1e21-0f76-4965-9800-8b0bae289f21 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:55.326374 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:55.334247 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23ee14-63e2-4739-a50e-4fc806223305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:55.334842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23ee14-63e2-4739-a50e-4fc806223305 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:55.336365 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23ee14-63e2-4739-a50e-4fc806223305 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:55.336365 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23ee14-63e2-4739-a50e-4fc806223305 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:55.398503 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c23ee14-63e2-4739-a50e-4fc806223305 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:55.399941 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c23ee14-63e2-4739-a50e-4fc806223305 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:55.491625 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 420/837] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:33:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 03:33:55.497306 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566e8c1-a8a9-48d0-ae3b-8be070ae1942 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:55.587200 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:55.593747 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566e8c1-a8a9-48d0-ae3b-8be070ae1942 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:55.625065 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4615f3d-a9e6-4e72-88a1-5b3daebf7c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:55.625684 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4615f3d-a9e6-4e72-88a1-5b3daebf7c64 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:55.626154 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4615f3d-a9e6-4e72-88a1-5b3daebf7c64 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:55.626637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4615f3d-a9e6-4e72-88a1-5b3daebf7c64 None admin] PATH_INFO: `/v3/projects/4b2d78dfa47b4129a0cd63d58a3c4554` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:55.628143 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4615f3d-a9e6-4e72-88a1-5b3daebf7c64 None admin] RBAC: Authorizing `identity:get_project(project_id=4b2d78dfa47b4129a0cd63d58a3c4554)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:55.629559 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4615f3d-a9e6-4e72-88a1-5b3daebf7c64 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:55.631653 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 418/838] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:33:55 2022] GET /identity/v3/projects/4b2d78dfa47b4129a0cd63d58a3c4554 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:55.637552 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23ee14-63e2-4739-a50e-4fc806223305 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:55.663971 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:55.731815 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c23ee14-63e2-4739-a50e-4fc806223305 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:55.763745 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab2047-cda5-4980-9a5a-7acac27a09be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:55.764351 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab2047-cda5-4980-9a5a-7acac27a09be None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:55.764627 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab2047-cda5-4980-9a5a-7acac27a09be None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:55.764895 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab2047-cda5-4980-9a5a-7acac27a09be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:55.765602 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab2047-cda5-4980-9a5a-7acac27a09be None admin] RBAC: Adding query filter params (role.id=611120afb3a544d0b5161da1b769a2f8, scope.project.id=4b2d78dfa47b4129a0cd63d58a3c4554, user.id=ffc48e6f1c1449d39ef5649faf585bd1) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:33:55.765907 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab2047-cda5-4980-9a5a-7acac27a09be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:55.767308 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab2047-cda5-4980-9a5a-7acac27a09be None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:55.773338 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 421/839] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Oct 17 03:33:55 2022] GET /identity/v3/role_assignments?user.id=ffc48e6f1c1449d39ef5649faf585bd1&scope.project.id=4b2d78dfa47b4129a0cd63d58a3c4554&role.id=611120afb3a544d0b5161da1b769a2f8 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:57.924219 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f01e1-d9a0-4701-adda-c5636b2201e9 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:57.924763 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f01e1-d9a0-4701-adda-c5636b2201e9 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:57.924763 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f01e1-d9a0-4701-adda-c5636b2201e9 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:57.926086 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 419/840] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:33:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:33:57.931242 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010bf05-841c-4dc5-949e-a7962410bccf None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:57.931615 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010bf05-841c-4dc5-949e-a7962410bccf None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:57.931832 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010bf05-841c-4dc5-949e-a7962410bccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:57.949989 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-c010bf05-841c-4dc5-949e-a7962410bccf None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:33:57.951107 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c010bf05-841c-4dc5-949e-a7962410bccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:33:58.014908 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 422/841] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:33:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:33:58.283423 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f01e1-d9a0-4701-adda-c5636b2201e9 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:58.381257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f01e1-d9a0-4701-adda-c5636b2201e9 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:58.413724 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f602dcd2-e473-4b7d-98f3-77ba40f9cee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:58.414348 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602dcd2-e473-4b7d-98f3-77ba40f9cee3 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:58.414630 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602dcd2-e473-4b7d-98f3-77ba40f9cee3 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:58.414907 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602dcd2-e473-4b7d-98f3-77ba40f9cee3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:58.416040 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f602dcd2-e473-4b7d-98f3-77ba40f9cee3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:58.417459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f602dcd2-e473-4b7d-98f3-77ba40f9cee3 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:58.419004 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 420/842] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:33:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:33:58.424526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010bf05-841c-4dc5-949e-a7962410bccf None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:33:58.448188 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:33:58.520617 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010bf05-841c-4dc5-949e-a7962410bccf None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:33:58.554244 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1c3d0-ea2d-487d-b1e9-b3c4c7ea71a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:33:58.554790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1c3d0-ea2d-487d-b1e9-b3c4c7ea71a1 None admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:33:58.555065 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1c3d0-ea2d-487d-b1e9-b3c4c7ea71a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:33:58.555333 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1c3d0-ea2d-487d-b1e9-b3c4c7ea71a1 None admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:33:58.556007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa1c3d0-ea2d-487d-b1e9-b3c4c7ea71a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:33:58.557418 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa1c3d0-ea2d-487d-b1e9-b3c4c7ea71a1 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:33:58.643386 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 423/843] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Oct 17 03:33:58 2022] POST /identity/v3/users => generated 286 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:34:00.791019 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73d151-c05c-43a7-9604-141469a316c4 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:00.791565 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73d151-c05c-43a7-9604-141469a316c4 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:00.791565 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73d151-c05c-43a7-9604-141469a316c4 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:00.793005 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 421/844] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:34:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:34:00.798277 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce6b60-8ff0-440e-873e-bfc58f854605 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:00.798677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce6b60-8ff0-440e-873e-bfc58f854605 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:00.798818 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce6b60-8ff0-440e-873e-bfc58f854605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:00.816431 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-9dce6b60-8ff0-440e-873e-bfc58f854605 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:34:00.817614 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-9dce6b60-8ff0-440e-873e-bfc58f854605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:34:00.881718 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 424/845] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:34:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:34:01.148693 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d73d151-c05c-43a7-9604-141469a316c4 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:01.178606 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:01.248326 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d73d151-c05c-43a7-9604-141469a316c4 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:01.280729 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2b9ae-7607-41cb-9cdc-112cc82b9f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:01.281344 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2b9ae-7607-41cb-9cdc-112cc82b9f68 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:01.281624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2b9ae-7607-41cb-9cdc-112cc82b9f68 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:01.281896 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2b9ae-7607-41cb-9cdc-112cc82b9f68 None admin] PATH_INFO: `/v3/roles/609a4bf154ff4dcabb408b3789d28983` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:01.283247 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2b9ae-7607-41cb-9cdc-112cc82b9f68 None admin] RBAC: Authorizing `identity:get_role(role_id=609a4bf154ff4dcabb408b3789d28983)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:01.284616 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2b9ae-7607-41cb-9cdc-112cc82b9f68 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:01.285906 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 422/846] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:34:01 2022] GET /identity/v3/roles/609a4bf154ff4dcabb408b3789d28983 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:34:01.291644 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce6b60-8ff0-440e-873e-bfc58f854605 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:01.301525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:01.387078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce6b60-8ff0-440e-873e-bfc58f854605 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:01.418712 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10613ef3-c0cd-4d54-a98b-f179011f49ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:01.419273 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10613ef3-c0cd-4d54-a98b-f179011f49ae None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:01.419559 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10613ef3-c0cd-4d54-a98b-f179011f49ae None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:01.419845 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10613ef3-c0cd-4d54-a98b-f179011f49ae None admin] PATH_INFO: `/v3/users/187fab3ef3fd4346b1d26a30b9a1cf48` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:01.432641 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10613ef3-c0cd-4d54-a98b-f179011f49ae None admin] RBAC: Authorizing `identity:get_user(user_id=187fab3ef3fd4346b1d26a30b9a1cf48)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:01.434042 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10613ef3-c0cd-4d54-a98b-f179011f49ae None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:01.435586 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 425/847] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:34:01 2022] GET /identity/v3/users/187fab3ef3fd4346b1d26a30b9a1cf48 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:34:01.441163 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2b9ae-7607-41cb-9cdc-112cc82b9f68 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:01.534315 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:01.538485 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2b9ae-7607-41cb-9cdc-112cc82b9f68 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:01.570881 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc6bea-65ef-495d-957d-e9ead04748fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:01.571490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc6bea-65ef-495d-957d-e9ead04748fb None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:01.571794 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc6bea-65ef-495d-957d-e9ead04748fb None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:01.572104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc6bea-65ef-495d-957d-e9ead04748fb None admin] PATH_INFO: `/v3/projects/4b2d78dfa47b4129a0cd63d58a3c4554` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:01.573486 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc6bea-65ef-495d-957d-e9ead04748fb None admin] RBAC: Authorizing `identity:get_project(project_id=4b2d78dfa47b4129a0cd63d58a3c4554)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:01.575044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc6bea-65ef-495d-957d-e9ead04748fb None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:01.576954 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 423/848] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:34:01 2022] GET /identity/v3/projects/4b2d78dfa47b4129a0cd63d58a3c4554 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:34:01.582598 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10613ef3-c0cd-4d54-a98b-f179011f49ae None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:01.644746 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:01.677722 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10613ef3-c0cd-4d54-a98b-f179011f49ae None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:01.710289 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd1ee4d-08ea-44ef-bdc7-b4b46eb7b520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:01.710935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1ee4d-08ea-44ef-bdc7-b4b46eb7b520 None admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:01.711213 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1ee4d-08ea-44ef-bdc7-b4b46eb7b520 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:01.711482 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd1ee4d-08ea-44ef-bdc7-b4b46eb7b520 None admin] PATH_INFO: `/v3/projects/4b2d78dfa47b4129a0cd63d58a3c4554/users/187fab3ef3fd4346b1d26a30b9a1cf48/roles/609a4bf154ff4dcabb408b3789d28983` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:01.713867 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1ee4d-08ea-44ef-bdc7-b4b46eb7b520 None admin] RBAC: Authorizing `identity:create_grant(project_id=4b2d78dfa47b4129a0cd63d58a3c4554, user_id=187fab3ef3fd4346b1d26a30b9a1cf48, role_id=609a4bf154ff4dcabb408b3789d28983)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:01.715278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd1ee4d-08ea-44ef-bdc7-b4b46eb7b520 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:01.824498 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 426/849] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Oct 17 03:34:01 2022] PUT /identity/v3/projects/4b2d78dfa47b4129a0cd63d58a3c4554/users/187fab3ef3fd4346b1d26a30b9a1cf48/roles/609a4bf154ff4dcabb408b3789d28983 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:34:03.920224 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28bf383-306e-45f4-8941-f98a152df0a7 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:03.920743 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28bf383-306e-45f4-8941-f98a152df0a7 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:03.920743 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28bf383-306e-45f4-8941-f98a152df0a7 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:03.922093 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 424/850] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:34:03.927293 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c0da2-47c5-4159-81ce-74cd151aa77c None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:03.927693 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c0da2-47c5-4159-81ce-74cd151aa77c None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:03.927847 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c0da2-47c5-4159-81ce-74cd151aa77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:03.945926 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-c65c0da2-47c5-4159-81ce-74cd151aa77c None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:34:03.947047 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c65c0da2-47c5-4159-81ce-74cd151aa77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:34:04.010158 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 427/851] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:34:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:34:04.274143 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28bf383-306e-45f4-8941-f98a152df0a7 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:04.371755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28bf383-306e-45f4-8941-f98a152df0a7 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:04.404566 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940a181-497f-4715-a68e-20181c56ed47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:04.405184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940a181-497f-4715-a68e-20181c56ed47 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:04.405465 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940a181-497f-4715-a68e-20181c56ed47 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:04.405739 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940a181-497f-4715-a68e-20181c56ed47 None admin] PATH_INFO: `/v3/roles/609a4bf154ff4dcabb408b3789d28983` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:04.407114 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3940a181-497f-4715-a68e-20181c56ed47 None admin] RBAC: Authorizing `identity:get_role(role_id=609a4bf154ff4dcabb408b3789d28983)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:04.408492 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3940a181-497f-4715-a68e-20181c56ed47 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:04.409780 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 425/852] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:34:04 2022] GET /identity/v3/roles/609a4bf154ff4dcabb408b3789d28983 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:34:04.415311 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c0da2-47c5-4159-81ce-74cd151aa77c None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:04.468337 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:04.509619 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c0da2-47c5-4159-81ce-74cd151aa77c None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:04.542166 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27124b9-770b-4b6f-99f2-c76abe0a4974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:04.542742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27124b9-770b-4b6f-99f2-c76abe0a4974 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:04.543020 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27124b9-770b-4b6f-99f2-c76abe0a4974 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:04.543290 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27124b9-770b-4b6f-99f2-c76abe0a4974 None admin] PATH_INFO: `/v3/users/187fab3ef3fd4346b1d26a30b9a1cf48` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:04.544369 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27124b9-770b-4b6f-99f2-c76abe0a4974 None admin] RBAC: Authorizing `identity:get_user(user_id=187fab3ef3fd4346b1d26a30b9a1cf48)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:04.545772 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27124b9-770b-4b6f-99f2-c76abe0a4974 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:04.547305 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 428/853] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:34:04 2022] GET /identity/v3/users/187fab3ef3fd4346b1d26a30b9a1cf48 => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:34:04.553528 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940a181-497f-4715-a68e-20181c56ed47 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:04.586228 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:04.650329 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940a181-497f-4715-a68e-20181c56ed47 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:04.684815 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc62bc2-7ecd-45ea-bd7e-bd0abd048be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:04.685421 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc62bc2-7ecd-45ea-bd7e-bd0abd048be7 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:04.685704 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc62bc2-7ecd-45ea-bd7e-bd0abd048be7 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:04.685982 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc62bc2-7ecd-45ea-bd7e-bd0abd048be7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:04.751720 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc62bc2-7ecd-45ea-bd7e-bd0abd048be7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:04.753197 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc62bc2-7ecd-45ea-bd7e-bd0abd048be7 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:04.784142 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:04.848460 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 426/854] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:34:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 03:34:04.855033 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27124b9-770b-4b6f-99f2-c76abe0a4974 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:04.948231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27124b9-770b-4b6f-99f2-c76abe0a4974 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:04.970554 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:04.981138 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7486513-55e7-426e-8f92-e2a0e0c6f945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:04.982004 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7486513-55e7-426e-8f92-e2a0e0c6f945 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:04.982099 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7486513-55e7-426e-8f92-e2a0e0c6f945 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:04.982340 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7486513-55e7-426e-8f92-e2a0e0c6f945 None admin] PATH_INFO: `/v3/projects/4b2d78dfa47b4129a0cd63d58a3c4554` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:04.983448 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7486513-55e7-426e-8f92-e2a0e0c6f945 None admin] RBAC: Authorizing `identity:get_project(project_id=4b2d78dfa47b4129a0cd63d58a3c4554)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:04.984907 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7486513-55e7-426e-8f92-e2a0e0c6f945 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:04.986507 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 429/855] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:34:04 2022] GET /identity/v3/projects/4b2d78dfa47b4129a0cd63d58a3c4554 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:34:04.992778 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc62bc2-7ecd-45ea-bd7e-bd0abd048be7 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:05.089049 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc62bc2-7ecd-45ea-bd7e-bd0abd048be7 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:05.121461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55087c-44aa-4f4d-8a79-95ab42f117d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:05.121999 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55087c-44aa-4f4d-8a79-95ab42f117d0 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:05.122281 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55087c-44aa-4f4d-8a79-95ab42f117d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:05.122557 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55087c-44aa-4f4d-8a79-95ab42f117d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:05.123276 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55087c-44aa-4f4d-8a79-95ab42f117d0 None admin] RBAC: Adding query filter params (role.id=609a4bf154ff4dcabb408b3789d28983, scope.project.id=4b2d78dfa47b4129a0cd63d58a3c4554, user.id=187fab3ef3fd4346b1d26a30b9a1cf48) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:34:05.123585 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55087c-44aa-4f4d-8a79-95ab42f117d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:05.125023 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55087c-44aa-4f4d-8a79-95ab42f117d0 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:05.126752 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:05.131103 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 427/856] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Oct 17 03:34:04 2022] GET /identity/v3/role_assignments?user.id=187fab3ef3fd4346b1d26a30b9a1cf48&scope.project.id=4b2d78dfa47b4129a0cd63d58a3c4554&role.id=609a4bf154ff4dcabb408b3789d28983 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:34:07.231684 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd04ca2-bfeb-446b-9819-3b8b28840d10 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:07.232260 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd04ca2-bfeb-446b-9819-3b8b28840d10 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:07.232260 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd04ca2-bfeb-446b-9819-3b8b28840d10 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:07.233593 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 430/857] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:34:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:34:07.238775 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ab075-7106-40d4-b277-14a1433f0b4c None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:07.239707 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ab075-7106-40d4-b277-14a1433f0b4c None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:07.239826 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ab075-7106-40d4-b277-14a1433f0b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:07.258203 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-d33ab075-7106-40d4-b277-14a1433f0b4c None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:34:07.259378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-d33ab075-7106-40d4-b277-14a1433f0b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:34:07.323517 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 428/858] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:34:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:34:07.592885 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd04ca2-bfeb-446b-9819-3b8b28840d10 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:07.689454 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd04ca2-bfeb-446b-9819-3b8b28840d10 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:07.721538 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ed1953-7317-4dad-94ba-5506c2eefe99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:07.722152 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed1953-7317-4dad-94ba-5506c2eefe99 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:07.722429 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed1953-7317-4dad-94ba-5506c2eefe99 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:07.722697 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ed1953-7317-4dad-94ba-5506c2eefe99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:07.724323 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ed1953-7317-4dad-94ba-5506c2eefe99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:07.725722 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ed1953-7317-4dad-94ba-5506c2eefe99 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:07.727246 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 431/859] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:34:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:34:07.732977 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ab075-7106-40d4-b277-14a1433f0b4c None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:07.828619 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ab075-7106-40d4-b277-14a1433f0b4c None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:07.854968 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:07.862343 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505072eb-5609-4dca-a6cf-7436fb9892ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:07.862936 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505072eb-5609-4dca-a6cf-7436fb9892ee None admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:07.863217 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505072eb-5609-4dca-a6cf-7436fb9892ee None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:07.863490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505072eb-5609-4dca-a6cf-7436fb9892ee None admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:07.864197 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505072eb-5609-4dca-a6cf-7436fb9892ee None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:07.865609 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505072eb-5609-4dca-a6cf-7436fb9892ee None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:07.950488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 429/860] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Oct 17 03:34:07 2022] POST /identity/v3/projects => generated 311 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:34:10.091764 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c75769-965f-47c5-aa45-2e535374fb00 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:10.092352 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c75769-965f-47c5-aa45-2e535374fb00 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:10.092352 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c75769-965f-47c5-aa45-2e535374fb00 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:10.093635 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 432/861] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:34:10.098769 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c15d0-a58b-4f06-b477-19f166e56024 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:10.099180 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c15d0-a58b-4f06-b477-19f166e56024 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:10.099323 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8c15d0-a58b-4f06-b477-19f166e56024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:10.117318 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-3f8c15d0-a58b-4f06-b477-19f166e56024 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:34:10.118498 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-3f8c15d0-a58b-4f06-b477-19f166e56024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:34:10.182298 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 430/862] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:34:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:34:10.455231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c75769-965f-47c5-aa45-2e535374fb00 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:10.513564 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:10.553231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c75769-965f-47c5-aa45-2e535374fb00 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:10.585478 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fe7e23-597a-479b-89ff-15d50c102994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:10.586087 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe7e23-597a-479b-89ff-15d50c102994 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:10.586367 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe7e23-597a-479b-89ff-15d50c102994 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:10.586637 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fe7e23-597a-479b-89ff-15d50c102994 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:10.587999 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe7e23-597a-479b-89ff-15d50c102994 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:10.589403 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fe7e23-597a-479b-89ff-15d50c102994 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:10.590919 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 433/863] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:34:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:34:10.596536 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c15d0-a58b-4f06-b477-19f166e56024 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:10.692897 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8c15d0-a58b-4f06-b477-19f166e56024 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:10.703470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:10.726774 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413db508-333b-468a-8c5e-2a0fba6e755c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:10.727371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413db508-333b-468a-8c5e-2a0fba6e755c None admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:10.727651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413db508-333b-468a-8c5e-2a0fba6e755c None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:10.727974 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413db508-333b-468a-8c5e-2a0fba6e755c None admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:10.728635 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413db508-333b-468a-8c5e-2a0fba6e755c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:10.730052 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413db508-333b-468a-8c5e-2a0fba6e755c None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:10.857790 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 431/864] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Oct 17 03:34:10 2022] POST /identity/v3/users => generated 286 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:34:12.986257 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956cb0da-3a5b-4ad8-b479-26d9ebc99a6f None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:12.986789 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956cb0da-3a5b-4ad8-b479-26d9ebc99a6f None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:12.986789 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956cb0da-3a5b-4ad8-b479-26d9ebc99a6f None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:12.988181 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 434/865] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:34:12.993236 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3cc81f-62ee-447e-a9d7-0fa83c3cb167 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:12.993678 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3cc81f-62ee-447e-a9d7-0fa83c3cb167 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:12.993771 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3cc81f-62ee-447e-a9d7-0fa83c3cb167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:13.012032 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-3b3cc81f-62ee-447e-a9d7-0fa83c3cb167 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:34:13.013206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-3b3cc81f-62ee-447e-a9d7-0fa83c3cb167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:34:13.076862 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 432/866] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:34:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 17 03:34:13.344328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956cb0da-3a5b-4ad8-b479-26d9ebc99a6f None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:13.441617 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956cb0da-3a5b-4ad8-b479-26d9ebc99a6f None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:13.467128 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:13.475016 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fe2d3-95b2-4912-b242-902938b7363f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:13.475680 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fe2d3-95b2-4912-b242-902938b7363f None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:13.476005 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fe2d3-95b2-4912-b242-902938b7363f None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:13.476283 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2fe2d3-95b2-4912-b242-902938b7363f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:13.485851 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2fe2d3-95b2-4912-b242-902938b7363f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:34:13.491652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-ab2fe2d3-95b2-4912-b242-902938b7363f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:34:13.491652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:34:13.491652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:34:13.491652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:34:13.491652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:34:13.491652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:34:13.491652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:34:13.491652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:34:13.491652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:34:13.491652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:34:13.491652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:34:13.491652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:34:13.491652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:34:13.491652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:34:13.491652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:34:13.491652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:34:13.492760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:34:13.492760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:34:13.492760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:34:13.492760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:34:13.492760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:34:13.492760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:34:13.492760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:34:13.492760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:34:13.492760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:34:13.492760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:34:13.492760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:34:13.492760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:34:13.492760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:34:13.492760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:34:13.492760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:34:13.492760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:34:13.493724 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:34:13.493724 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:34:13.493724 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:34:13.493724 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:34:13.493724 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:34:13.493724 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:34:13.493724 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:34:13.493724 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:34:13.493724 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:34:13.493724 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:34:13.493724 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:34:13.493724 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:34:13.493724 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 435/867] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Oct 17 03:34:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:34:13.498597 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3cc81f-62ee-447e-a9d7-0fa83c3cb167 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:13.588077 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:13.593563 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3cc81f-62ee-447e-a9d7-0fa83c3cb167 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:13.627672 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a8e48-67b3-4b7c-82f1-e1b418ad435a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:13.628243 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a8e48-67b3-4b7c-82f1-e1b418ad435a None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:13.628524 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a8e48-67b3-4b7c-82f1-e1b418ad435a None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:13.628795 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799a8e48-67b3-4b7c-82f1-e1b418ad435a None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:13.629477 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a8e48-67b3-4b7c-82f1-e1b418ad435a None admin] RBAC: Adding query filter params (name=admin) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:34:13.629780 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a8e48-67b3-4b7c-82f1-e1b418ad435a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:13.631132 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799a8e48-67b3-4b7c-82f1-e1b418ad435a None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:13.638849 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 433/868] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:34:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:34:13.644662 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fe2d3-95b2-4912-b242-902938b7363f None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:13.744406 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2fe2d3-95b2-4912-b242-902938b7363f None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:13.808405 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6c0c1-3789-4945-94be-31fb824c6467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:13.809031 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6c0c1-3789-4945-94be-31fb824c6467 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:13.809313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6c0c1-3789-4945-94be-31fb824c6467 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:13.809594 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6c0c1-3789-4945-94be-31fb824c6467 None admin] PATH_INFO: `/v3/users/d64f0a9b4abd4787af6245ff2932646c` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:13.822120 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f6c0c1-3789-4945-94be-31fb824c6467 None admin] RBAC: Authorizing `identity:get_user(user_id=d64f0a9b4abd4787af6245ff2932646c)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:13.823535 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f6c0c1-3789-4945-94be-31fb824c6467 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:13.825207 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 436/869] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:34:13 2022] GET /identity/v3/users/d64f0a9b4abd4787af6245ff2932646c => generated 286 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:34:13.831392 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a8e48-67b3-4b7c-82f1-e1b418ad435a None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:13.927958 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799a8e48-67b3-4b7c-82f1-e1b418ad435a None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:13.960207 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eec4476-fb3b-4255-9afe-4fba0a42613c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:13.960827 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec4476-fb3b-4255-9afe-4fba0a42613c None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:13.961113 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec4476-fb3b-4255-9afe-4fba0a42613c None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:13.961391 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec4476-fb3b-4255-9afe-4fba0a42613c None admin] PATH_INFO: `/v3/projects/50e67faca06246cba960575e667a384c` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:13.962760 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eec4476-fb3b-4255-9afe-4fba0a42613c None admin] RBAC: Authorizing `identity:get_project(project_id=50e67faca06246cba960575e667a384c)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:13.964218 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eec4476-fb3b-4255-9afe-4fba0a42613c None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:13.965832 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 434/870] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:34:13 2022] GET /identity/v3/projects/50e67faca06246cba960575e667a384c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:34:13.971627 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6c0c1-3789-4945-94be-31fb824c6467 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:14.017545 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:14.068052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6c0c1-3789-4945-94be-31fb824c6467 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:14.101025 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06932e-d313-491c-9576-2038fba6f06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:14.101691 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06932e-d313-491c-9576-2038fba6f06b None admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:14.101971 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06932e-d313-491c-9576-2038fba6f06b None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:14.102239 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06932e-d313-491c-9576-2038fba6f06b None admin] PATH_INFO: `/v3/projects/50e67faca06246cba960575e667a384c/users/d64f0a9b4abd4787af6245ff2932646c/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:14.104126 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c06932e-d313-491c-9576-2038fba6f06b None admin] RBAC: Authorizing `identity:create_grant(project_id=50e67faca06246cba960575e667a384c, user_id=d64f0a9b4abd4787af6245ff2932646c, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:14.105529 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c06932e-d313-491c-9576-2038fba6f06b None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:14.202039 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 437/871] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Oct 17 03:34:13 2022] PUT /identity/v3/projects/50e67faca06246cba960575e667a384c/users/d64f0a9b4abd4787af6245ff2932646c/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:34:16.284629 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551aa46-646d-4a64-9b1f-7a02432e400d None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:16.285190 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551aa46-646d-4a64-9b1f-7a02432e400d None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:16.285190 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8551aa46-646d-4a64-9b1f-7a02432e400d None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:16.286538 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 435/872] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:34:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:34:16.291585 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad18fee-8a2c-4e58-836b-14db9eb31d48 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:16.291980 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad18fee-8a2c-4e58-836b-14db9eb31d48 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:16.292158 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad18fee-8a2c-4e58-836b-14db9eb31d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:16.311199 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-8ad18fee-8a2c-4e58-836b-14db9eb31d48 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:34:16.312395 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-8ad18fee-8a2c-4e58-836b-14db9eb31d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:34:16.374827 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 438/873] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:34:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:34:16.643049 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8551aa46-646d-4a64-9b1f-7a02432e400d None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:16.670608 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:16.745154 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8551aa46-646d-4a64-9b1f-7a02432e400d None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:16.778492 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4116b531-f5f4-47cc-891d-d903a23926a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:16.779121 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116b531-f5f4-47cc-891d-d903a23926a8 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:16.779404 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116b531-f5f4-47cc-891d-d903a23926a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:16.779706 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116b531-f5f4-47cc-891d-d903a23926a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:16.790208 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4116b531-f5f4-47cc-891d-d903a23926a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:34:16.795012 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-4116b531-f5f4-47cc-891d-d903a23926a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:34:16.795012 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:34:16.795012 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:34:16.795012 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:34:16.795012 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:34:16.795012 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:34:16.795012 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:34:16.795012 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:34:16.795012 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:34:16.795012 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:34:16.795012 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:34:16.795012 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:34:16.795012 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:34:16.795012 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:34:16.795012 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:34:16.795012 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:34:16.796118 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:34:16.796118 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:34:16.796118 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:34:16.796118 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:34:16.796118 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:34:16.796118 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:34:16.796118 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:34:16.796118 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:34:16.796118 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:34:16.796118 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:34:16.796118 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:34:16.796118 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:34:16.796118 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:34:16.796118 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:34:16.796118 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:34:16.796118 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:34:16.797176 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:34:16.797176 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:34:16.797176 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:34:16.797176 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:34:16.797176 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:34:16.797176 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:34:16.797176 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:34:16.797176 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:34:16.797176 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:34:16.797176 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:34:16.797176 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:34:16.797176 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:34:16.797176 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 436/874] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Oct 17 03:34:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:34:16.802892 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad18fee-8a2c-4e58-836b-14db9eb31d48 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:16.835997 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:16.897646 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad18fee-8a2c-4e58-836b-14db9eb31d48 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:16.930506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638369ae-37ff-4d18-a384-2da92047292b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:16.931098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638369ae-37ff-4d18-a384-2da92047292b None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:16.931378 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638369ae-37ff-4d18-a384-2da92047292b None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:16.931648 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638369ae-37ff-4d18-a384-2da92047292b None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:16.932329 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638369ae-37ff-4d18-a384-2da92047292b None admin] RBAC: Adding query filter params (name=admin) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:34:16.932628 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638369ae-37ff-4d18-a384-2da92047292b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:16.933969 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638369ae-37ff-4d18-a384-2da92047292b None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:16.941622 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 439/875] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:34:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:34:16.947476 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4116b531-f5f4-47cc-891d-d903a23926a8 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:17.029709 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:17.045192 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4116b531-f5f4-47cc-891d-d903a23926a8 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:17.078595 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f0a79-4781-4e62-a0fd-911b38ab9f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:17.079215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f0a79-4781-4e62-a0fd-911b38ab9f03 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:17.079493 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f0a79-4781-4e62-a0fd-911b38ab9f03 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:17.079783 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f0a79-4781-4e62-a0fd-911b38ab9f03 None admin] PATH_INFO: `/v3/users/d64f0a9b4abd4787af6245ff2932646c` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:17.080877 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f0a79-4781-4e62-a0fd-911b38ab9f03 None admin] RBAC: Authorizing `identity:get_user(user_id=d64f0a9b4abd4787af6245ff2932646c)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:17.082784 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f0a79-4781-4e62-a0fd-911b38ab9f03 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:17.084376 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 437/876] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:34:16 2022] GET /identity/v3/users/d64f0a9b4abd4787af6245ff2932646c => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:34:17.090407 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638369ae-37ff-4d18-a384-2da92047292b None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:17.181926 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:17.185302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638369ae-37ff-4d18-a384-2da92047292b None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:17.217578 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87661203-e1f8-4415-b93c-a34b0a66fd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:17.218171 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87661203-e1f8-4415-b93c-a34b0a66fd53 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:17.218452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87661203-e1f8-4415-b93c-a34b0a66fd53 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:17.218724 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87661203-e1f8-4415-b93c-a34b0a66fd53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:17.282406 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87661203-e1f8-4415-b93c-a34b0a66fd53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:17.283848 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87661203-e1f8-4415-b93c-a34b0a66fd53 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:17.372091 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:17.376351 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 440/877] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:34:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 03:34:17.382652 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f0a79-4781-4e62-a0fd-911b38ab9f03 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:17.479088 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f0a79-4781-4e62-a0fd-911b38ab9f03 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:17.511378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5258739c-ceb4-464b-82e6-acbbfb54feda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:17.512014 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258739c-ceb4-464b-82e6-acbbfb54feda None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:17.512290 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258739c-ceb4-464b-82e6-acbbfb54feda None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:17.512558 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5258739c-ceb4-464b-82e6-acbbfb54feda None admin] PATH_INFO: `/v3/projects/50e67faca06246cba960575e667a384c` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:17.513647 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5258739c-ceb4-464b-82e6-acbbfb54feda None admin] RBAC: Authorizing `identity:get_project(project_id=50e67faca06246cba960575e667a384c)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:17.515056 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5258739c-ceb4-464b-82e6-acbbfb54feda None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:17.516670 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 438/878] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:34:17 2022] GET /identity/v3/projects/50e67faca06246cba960575e667a384c => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:34:17.523364 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87661203-e1f8-4415-b93c-a34b0a66fd53 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:17.618761 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87661203-e1f8-4415-b93c-a34b0a66fd53 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:17.651307 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2cf064-fdde-42fd-b585-907bc5a3a77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:17.651917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2cf064-fdde-42fd-b585-907bc5a3a77b None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:17.652198 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2cf064-fdde-42fd-b585-907bc5a3a77b None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:17.652472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2cf064-fdde-42fd-b585-907bc5a3a77b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:17.653197 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2cf064-fdde-42fd-b585-907bc5a3a77b None admin] RBAC: Adding query filter params (role.id=611120afb3a544d0b5161da1b769a2f8, scope.project.id=50e67faca06246cba960575e667a384c, user.id=d64f0a9b4abd4787af6245ff2932646c) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:34:17.653505 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2cf064-fdde-42fd-b585-907bc5a3a77b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:17.654907 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2cf064-fdde-42fd-b585-907bc5a3a77b None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:17.660498 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 441/879] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Oct 17 03:34:17 2022] GET /identity/v3/role_assignments?user.id=d64f0a9b4abd4787af6245ff2932646c&scope.project.id=50e67faca06246cba960575e667a384c&role.id=611120afb3a544d0b5161da1b769a2f8 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:34:19.799649 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21889e-9d8a-4d23-8773-c2b319aeff7c None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:19.800232 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21889e-9d8a-4d23-8773-c2b319aeff7c None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:19.800232 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21889e-9d8a-4d23-8773-c2b319aeff7c None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:19.801539 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 439/880] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:34:19.806663 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5c236-d776-4f29-8951-d0d309198519 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:19.807102 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5c236-d776-4f29-8951-d0d309198519 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:19.807198 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e5c236-d776-4f29-8951-d0d309198519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:19.825247 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-07e5c236-d776-4f29-8951-d0d309198519 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:34:19.826420 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-07e5c236-d776-4f29-8951-d0d309198519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:34:19.880467 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:19.890264 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 442/881] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:34:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:34:20.157014 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff21889e-9d8a-4d23-8773-c2b319aeff7c None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:20.181105 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:20.256118 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff21889e-9d8a-4d23-8773-c2b319aeff7c None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:20.288383 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e1ccc-a0c9-4418-9f82-98e3b917b02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:20.288972 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e1ccc-a0c9-4418-9f82-98e3b917b02a None admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:20.289252 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e1ccc-a0c9-4418-9f82-98e3b917b02a None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:20.289526 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e1ccc-a0c9-4418-9f82-98e3b917b02a None admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:20.290193 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e1ccc-a0c9-4418-9f82-98e3b917b02a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:20.291536 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e1ccc-a0c9-4418-9f82-98e3b917b02a None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:20.374466 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 440/882] 10.222.0.17 () {60 vars in 1138 bytes} [Mon Oct 17 03:34:20 2022] POST /identity/v3/domains => generated 267 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:34:22.505988 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f61b4-b0c4-45c9-8a02-138946a5c08e None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:22.506515 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f61b4-b0c4-45c9-8a02-138946a5c08e None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:22.506605 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f61b4-b0c4-45c9-8a02-138946a5c08e None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:22.507909 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 443/883] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:34:22.513238 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166fb1e-854f-4482-9ef0-704d43bd509a None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:22.513638 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166fb1e-854f-4482-9ef0-704d43bd509a None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:22.513787 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0166fb1e-854f-4482-9ef0-704d43bd509a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:22.531368 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-0166fb1e-854f-4482-9ef0-704d43bd509a None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:34:22.532566 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-0166fb1e-854f-4482-9ef0-704d43bd509a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:34:22.596553 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 441/884] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:34:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:34:22.861116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f61b4-b0c4-45c9-8a02-138946a5c08e None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:22.957920 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f61b4-b0c4-45c9-8a02-138946a5c08e None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:22.990276 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8713ab-0675-491a-bf57-4940da5c300c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:22.990886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8713ab-0675-491a-bf57-4940da5c300c None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:22.991164 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8713ab-0675-491a-bf57-4940da5c300c None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:22.991433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8713ab-0675-491a-bf57-4940da5c300c None admin] PATH_INFO: `/v3/domains/088b3903fa2845a3804b640026de120a` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:23.001330 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8713ab-0675-491a-bf57-4940da5c300c None admin] RBAC: Authorizing `identity:get_domain(domain_id=088b3903fa2845a3804b640026de120a)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:23.002748 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8713ab-0675-491a-bf57-4940da5c300c None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:23.004350 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 444/885] 10.222.0.17 () {56 vars in 1183 bytes} [Mon Oct 17 03:34:22 2022] GET /identity/v3/domains/088b3903fa2845a3804b640026de120a => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:34:23.010003 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166fb1e-854f-4482-9ef0-704d43bd509a None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:23.024163 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:23.107677 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0166fb1e-854f-4482-9ef0-704d43bd509a None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:23.140034 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a564d7-8d97-464f-8435-d51c89fdf5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:23.140580 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a564d7-8d97-464f-8435-d51c89fdf5b1 None admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:23.140862 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a564d7-8d97-464f-8435-d51c89fdf5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:23.141138 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a564d7-8d97-464f-8435-d51c89fdf5b1 None admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:23.141799 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a564d7-8d97-464f-8435-d51c89fdf5b1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:23.143218 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a564d7-8d97-464f-8435-d51c89fdf5b1 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:23.230564 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 442/886] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Oct 17 03:34:23 2022] POST /identity/v3/projects => generated 361 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:34:25.381633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9eaf8-a56a-4106-80b1-4a809079b1d5 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:25.382221 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9eaf8-a56a-4106-80b1-4a809079b1d5 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:25.382221 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9eaf8-a56a-4106-80b1-4a809079b1d5 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:25.383510 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 445/887] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:34:25.388644 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa949e0-7e1e-459e-802b-45ddc912f8ba None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:25.389028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa949e0-7e1e-459e-802b-45ddc912f8ba None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:25.389202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa949e0-7e1e-459e-802b-45ddc912f8ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:25.406973 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-eaa949e0-7e1e-459e-802b-45ddc912f8ba None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:34:25.408194 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-eaa949e0-7e1e-459e-802b-45ddc912f8ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:34:25.465130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:25.473534 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 443/888] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:34:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:34:25.740782 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9eaf8-a56a-4106-80b1-4a809079b1d5 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:25.793086 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:25.839164 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9eaf8-a56a-4106-80b1-4a809079b1d5 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:25.870565 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cd8293-576c-4e94-843c-f0c78a3fc3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:25.871174 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd8293-576c-4e94-843c-f0c78a3fc3fe None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:25.871451 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd8293-576c-4e94-843c-f0c78a3fc3fe None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:25.871727 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cd8293-576c-4e94-843c-f0c78a3fc3fe None admin] PATH_INFO: `/v3/domains/088b3903fa2845a3804b640026de120a` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:25.872835 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cd8293-576c-4e94-843c-f0c78a3fc3fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=088b3903fa2845a3804b640026de120a)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:25.874233 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cd8293-576c-4e94-843c-f0c78a3fc3fe None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:25.875760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 446/889] 10.222.0.17 () {56 vars in 1183 bytes} [Mon Oct 17 03:34:25 2022] GET /identity/v3/domains/088b3903fa2845a3804b640026de120a => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:34:25.881967 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa949e0-7e1e-459e-802b-45ddc912f8ba None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:25.977547 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa949e0-7e1e-459e-802b-45ddc912f8ba None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:26.011032 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f444301d-60be-42fe-ac76-fc6033257737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:26.011619 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444301d-60be-42fe-ac76-fc6033257737 None admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:26.011930 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444301d-60be-42fe-ac76-fc6033257737 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:26.012206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444301d-60be-42fe-ac76-fc6033257737 None admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:26.012854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f444301d-60be-42fe-ac76-fc6033257737 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:26.014265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f444301d-60be-42fe-ac76-fc6033257737 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:26.087955 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 444/890] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Oct 17 03:34:25 2022] POST /identity/v3/users => generated 311 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:34:28.215025 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e92310-9b55-418a-8835-22c7c2f6c7d2 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:28.215575 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e92310-9b55-418a-8835-22c7c2f6c7d2 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:28.215575 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e92310-9b55-418a-8835-22c7c2f6c7d2 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:28.216933 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 447/891] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:34:28.222610 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb765ee-aaf9-4fdf-a296-474c60c87e5e None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:28.222997 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb765ee-aaf9-4fdf-a296-474c60c87e5e None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:28.223160 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb765ee-aaf9-4fdf-a296-474c60c87e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:28.240752 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-ddb765ee-aaf9-4fdf-a296-474c60c87e5e None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:34:28.241921 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-ddb765ee-aaf9-4fdf-a296-474c60c87e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:34:28.286062 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:28.305479 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 445/892] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:34:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:34:28.571771 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e92310-9b55-418a-8835-22c7c2f6c7d2 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:28.668248 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e92310-9b55-418a-8835-22c7c2f6c7d2 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:28.687783 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:28.701590 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b366e-d2a1-4464-9df1-5745a6de3a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:28.702218 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b366e-d2a1-4464-9df1-5745a6de3a99 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:28.702496 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b366e-d2a1-4464-9df1-5745a6de3a99 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:28.702767 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b366e-d2a1-4464-9df1-5745a6de3a99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:28.713440 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b366e-d2a1-4464-9df1-5745a6de3a99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:34:28.718219 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-e69b366e-d2a1-4464-9df1-5745a6de3a99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:34:28.718219 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:34:28.718219 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:34:28.718219 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:34:28.718219 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:34:28.718219 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:34:28.718219 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:34:28.718219 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:34:28.718219 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:34:28.718219 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:34:28.718219 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:34:28.718219 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:34:28.718219 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:34:28.718219 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:34:28.718219 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:34:28.718219 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:34:28.719389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:34:28.719389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:34:28.719389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:34:28.719389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:34:28.719389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:34:28.719389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:34:28.719389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:34:28.719389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:34:28.719389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:34:28.719389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:34:28.719389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:34:28.719389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:34:28.719389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:34:28.719389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:34:28.719389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:34:28.719389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:34:28.720551 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:34:28.720551 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:34:28.720551 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:34:28.720551 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:34:28.720551 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:34:28.720551 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:34:28.720551 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:34:28.720551 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:34:28.720551 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:34:28.720551 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:34:28.720551 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:34:28.720551 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:34:28.720551 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 448/893] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Oct 17 03:34:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:34:28.725337 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb765ee-aaf9-4fdf-a296-474c60c87e5e None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:28.853758 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb765ee-aaf9-4fdf-a296-474c60c87e5e None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:28.886643 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4472ac-0af0-4dc7-859c-659eda15183a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:28.887244 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4472ac-0af0-4dc7-859c-659eda15183a None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:28.887526 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4472ac-0af0-4dc7-859c-659eda15183a None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:28.887828 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4472ac-0af0-4dc7-859c-659eda15183a None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:28.888484 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4472ac-0af0-4dc7-859c-659eda15183a None admin] RBAC: Adding query filter params (name=admin) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:34:28.888786 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4472ac-0af0-4dc7-859c-659eda15183a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:28.890144 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4472ac-0af0-4dc7-859c-659eda15183a None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:28.897878 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 446/894] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:34:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:34:28.903511 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b366e-d2a1-4464-9df1-5745a6de3a99 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:28.998824 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b366e-d2a1-4464-9df1-5745a6de3a99 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:29.031114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ccf6f-696b-48ca-bec5-b3a6d7f7202e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:29.031719 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ccf6f-696b-48ca-bec5-b3a6d7f7202e None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:29.032025 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ccf6f-696b-48ca-bec5-b3a6d7f7202e None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:29.032316 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ccf6f-696b-48ca-bec5-b3a6d7f7202e None admin] PATH_INFO: `/v3/users/82aa80ef0126444abcfdcc5e583b7c66` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:29.044649 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ccf6f-696b-48ca-bec5-b3a6d7f7202e None admin] RBAC: Authorizing `identity:get_user(user_id=82aa80ef0126444abcfdcc5e583b7c66)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:29.046042 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ccf6f-696b-48ca-bec5-b3a6d7f7202e None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:29.047575 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 449/895] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:34:28 2022] GET /identity/v3/users/82aa80ef0126444abcfdcc5e583b7c66 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:34:29.053771 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4472ac-0af0-4dc7-859c-659eda15183a None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:29.095957 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:29.151701 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4472ac-0af0-4dc7-859c-659eda15183a None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:29.183852 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459a44f-1e11-4c83-830f-58ebfed09b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:29.184470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459a44f-1e11-4c83-830f-58ebfed09b54 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:29.184749 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459a44f-1e11-4c83-830f-58ebfed09b54 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:29.185072 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459a44f-1e11-4c83-830f-58ebfed09b54 None admin] PATH_INFO: `/v3/projects/b91f15c557c24fbf8d2798697ce14870` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:29.186117 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3459a44f-1e11-4c83-830f-58ebfed09b54 None admin] RBAC: Authorizing `identity:get_project(project_id=b91f15c557c24fbf8d2798697ce14870)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:29.187522 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3459a44f-1e11-4c83-830f-58ebfed09b54 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:29.189131 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 447/896] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:34:29 2022] GET /identity/v3/projects/b91f15c557c24fbf8d2798697ce14870 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:34:29.195534 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ccf6f-696b-48ca-bec5-b3a6d7f7202e None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:29.234153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:29.291194 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ccf6f-696b-48ca-bec5-b3a6d7f7202e None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:29.323669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff7040-c928-40a9-abef-e2d12fedc859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:29.324354 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff7040-c928-40a9-abef-e2d12fedc859 None admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:29.324636 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff7040-c928-40a9-abef-e2d12fedc859 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:29.324910 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff7040-c928-40a9-abef-e2d12fedc859 None admin] PATH_INFO: `/v3/projects/b91f15c557c24fbf8d2798697ce14870/users/82aa80ef0126444abcfdcc5e583b7c66/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:29.326786 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ff7040-c928-40a9-abef-e2d12fedc859 None admin] RBAC: Authorizing `identity:create_grant(project_id=b91f15c557c24fbf8d2798697ce14870, user_id=82aa80ef0126444abcfdcc5e583b7c66, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:29.328322 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ff7040-c928-40a9-abef-e2d12fedc859 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:29.393639 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 450/897] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Oct 17 03:34:29 2022] PUT /identity/v3/projects/b91f15c557c24fbf8d2798697ce14870/users/82aa80ef0126444abcfdcc5e583b7c66/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:34:31.490028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bab634-fceb-4042-80ad-252e7b2aaaa1 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:31.490553 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bab634-fceb-4042-80ad-252e7b2aaaa1 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:31.490553 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bab634-fceb-4042-80ad-252e7b2aaaa1 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:31.491943 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 448/898] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:34:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:34:31.497354 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3e189-1b9c-4c2f-8ce2-f070c310e45c None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:31.497744 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3e189-1b9c-4c2f-8ce2-f070c310e45c None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:31.497899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3e189-1b9c-4c2f-8ce2-f070c310e45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:31.515448 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-b5c3e189-1b9c-4c2f-8ce2-f070c310e45c None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:34:31.516650 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-b5c3e189-1b9c-4c2f-8ce2-f070c310e45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:34:31.579414 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 451/899] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:34:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:34:31.844214 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bab634-fceb-4042-80ad-252e7b2aaaa1 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:31.942001 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bab634-fceb-4042-80ad-252e7b2aaaa1 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:31.952061 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:31.974286 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67171689-65b4-43d1-aa1c-c2c32567e027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:31.974896 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67171689-65b4-43d1-aa1c-c2c32567e027 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:31.975172 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67171689-65b4-43d1-aa1c-c2c32567e027 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:31.975440 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67171689-65b4-43d1-aa1c-c2c32567e027 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:31.985219 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67171689-65b4-43d1-aa1c-c2c32567e027 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:34:31.990757 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-67171689-65b4-43d1-aa1c-c2c32567e027 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:34:31.990757 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:34:31.990757 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:34:31.990757 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:34:31.990757 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:34:31.990757 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:34:31.990757 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:34:31.990757 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:34:31.990757 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:34:31.990757 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:34:31.990757 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:34:31.990757 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:34:31.990757 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:34:31.990757 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:34:31.990757 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:34:31.990757 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:34:31.991891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:34:31.991891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:34:31.991891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:34:31.991891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:34:31.991891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:34:31.991891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:34:31.991891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:34:31.991891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:34:31.991891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:34:31.991891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:34:31.991891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:34:31.991891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:34:31.991891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:34:31.991891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:34:31.991891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:34:31.991891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:34:31.992755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:34:31.992755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:34:31.992755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:34:31.992755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:34:31.992755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:34:31.992755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:34:31.992755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:34:31.992755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:34:31.992755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:34:31.992755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:34:31.992755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:34:31.992755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:34:31.992755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 449/900] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Oct 17 03:34:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:34:31.997784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3e189-1b9c-4c2f-8ce2-f070c310e45c None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:32.024623 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:32.093189 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3e189-1b9c-4c2f-8ce2-f070c310e45c None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:32.125317 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf549466-b495-486d-a251-ef6ffb48dafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:32.125859 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf549466-b495-486d-a251-ef6ffb48dafc None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:32.126140 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf549466-b495-486d-a251-ef6ffb48dafc None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:32.126414 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf549466-b495-486d-a251-ef6ffb48dafc None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:32.127065 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf549466-b495-486d-a251-ef6ffb48dafc None admin] RBAC: Adding query filter params (name=admin) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:34:32.127367 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf549466-b495-486d-a251-ef6ffb48dafc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:32.129340 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf549466-b495-486d-a251-ef6ffb48dafc None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:32.137025 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 452/901] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:34:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:34:32.142742 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67171689-65b4-43d1-aa1c-c2c32567e027 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:32.238706 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67171689-65b4-43d1-aa1c-c2c32567e027 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:32.270566 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f31b72-116a-450d-87a0-f16df9616f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:32.271179 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f31b72-116a-450d-87a0-f16df9616f10 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:32.271455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f31b72-116a-450d-87a0-f16df9616f10 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:32.271726 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f31b72-116a-450d-87a0-f16df9616f10 None admin] PATH_INFO: `/v3/users/82aa80ef0126444abcfdcc5e583b7c66` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:32.272850 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f31b72-116a-450d-87a0-f16df9616f10 None admin] RBAC: Authorizing `identity:get_user(user_id=82aa80ef0126444abcfdcc5e583b7c66)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:32.274264 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f31b72-116a-450d-87a0-f16df9616f10 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:32.275823 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 450/902] 10.222.0.17 () {56 vars in 1177 bytes} [Mon Oct 17 03:34:32 2022] GET /identity/v3/users/82aa80ef0126444abcfdcc5e583b7c66 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:34:32.281580 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf549466-b495-486d-a251-ef6ffb48dafc None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:32.368020 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:32.377037 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf549466-b495-486d-a251-ef6ffb48dafc None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:32.409765 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121f2f8-1918-4d95-b25c-c098d4859daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:32.410371 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121f2f8-1918-4d95-b25c-c098d4859daa None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:32.410648 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121f2f8-1918-4d95-b25c-c098d4859daa None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:32.410916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121f2f8-1918-4d95-b25c-c098d4859daa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:32.476198 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7121f2f8-1918-4d95-b25c-c098d4859daa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:32.477619 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7121f2f8-1918-4d95-b25c-c098d4859daa None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:32.560820 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:32.570555 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 453/903] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:34:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 03:34:32.576650 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f31b72-116a-450d-87a0-f16df9616f10 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:32.615933 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:32.673177 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f31b72-116a-450d-87a0-f16df9616f10 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:32.705381 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b99a1b2-86a2-4f17-9900-6bee012a9fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:32.706493 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99a1b2-86a2-4f17-9900-6bee012a9fc2 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:32.706766 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99a1b2-86a2-4f17-9900-6bee012a9fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:32.707033 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b99a1b2-86a2-4f17-9900-6bee012a9fc2 None admin] PATH_INFO: `/v3/projects/b91f15c557c24fbf8d2798697ce14870` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:32.708151 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99a1b2-86a2-4f17-9900-6bee012a9fc2 None admin] RBAC: Authorizing `identity:get_project(project_id=b91f15c557c24fbf8d2798697ce14870)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:32.709557 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b99a1b2-86a2-4f17-9900-6bee012a9fc2 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:32.711146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 451/904] 10.222.0.17 () {56 vars in 1186 bytes} [Mon Oct 17 03:34:32 2022] GET /identity/v3/projects/b91f15c557c24fbf8d2798697ce14870 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:34:32.717624 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121f2f8-1918-4d95-b25c-c098d4859daa None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:34:32.810427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121f2f8-1918-4d95-b25c-c098d4859daa None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:34:32.842316 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bbff59-e0ec-410c-8968-3d5b8cf891c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:34:32.842898 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbff59-e0ec-410c-8968-3d5b8cf891c4 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:34:32.843173 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbff59-e0ec-410c-8968-3d5b8cf891c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:34:32.843438 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbff59-e0ec-410c-8968-3d5b8cf891c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:34:32.844176 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bbff59-e0ec-410c-8968-3d5b8cf891c4 None admin] RBAC: Adding query filter params (role.id=611120afb3a544d0b5161da1b769a2f8, scope.project.id=b91f15c557c24fbf8d2798697ce14870, user.id=82aa80ef0126444abcfdcc5e583b7c66) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:34:32.844480 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bbff59-e0ec-410c-8968-3d5b8cf891c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:34:32.845874 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bbff59-e0ec-410c-8968-3d5b8cf891c4 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:34:32.851884 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 454/905] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Oct 17 03:34:32 2022] GET /identity/v3/role_assignments?user.id=82aa80ef0126444abcfdcc5e583b7c66&scope.project.id=b91f15c557c24fbf8d2798697ce14870&role.id=611120afb3a544d0b5161da1b769a2f8 => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:34:42.701261 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=47 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:34:42.849442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=73 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:35:44.757693 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790324e1-bea0-4528-b899-0f6e7e021951 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:44.758270 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790324e1-bea0-4528-b899-0f6e7e021951 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:44.758270 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790324e1-bea0-4528-b899-0f6e7e021951 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:44.759588 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 452/906] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:35:44.765043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9a34a-7dc8-4ac9-ae18-0b09309e7063 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:44.768504 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9a34a-7dc8-4ac9-ae18-0b09309e7063 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:44.768504 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda9a34a-7dc8-4ac9-ae18-0b09309e7063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:44.784503 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-cda9a34a-7dc8-4ac9-ae18-0b09309e7063 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:35:44.785703 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-cda9a34a-7dc8-4ac9-ae18-0b09309e7063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:35:44.850887 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 455/907] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:35:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:35:45.119107 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790324e1-bea0-4528-b899-0f6e7e021951 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:45.218182 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790324e1-bea0-4528-b899-0f6e7e021951 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:45.251625 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0e55c-f440-4e84-bd45-ca190d8b68e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:45.252294 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0e55c-f440-4e84-bd45-ca190d8b68e5 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:45.252613 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0e55c-f440-4e84-bd45-ca190d8b68e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:45.252890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0e55c-f440-4e84-bd45-ca190d8b68e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:45.262550 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c0e55c-f440-4e84-bd45-ca190d8b68e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:45.264010 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c0e55c-f440-4e84-bd45-ca190d8b68e5 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:45.279414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-37c0e55c-f440-4e84-bd45-ca190d8b68e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:35:45.279414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:35:45.279414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:35:45.279414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:35:45.279414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:35:45.279414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:35:45.279414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:35:45.279414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:35:45.279414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:35:45.279414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:35:45.279414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:35:45.279414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:35:45.279414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:35:45.279414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:35:45.279414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:35:45.279414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:35:45.280642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:35:45.280642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:35:45.280642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:35:45.280642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:35:45.280642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:35:45.280642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:35:45.280642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:35:45.280642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:35:45.280642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:35:45.280642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:35:45.280642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:35:45.280642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:35:45.280642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:35:45.280642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:35:45.280642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:35:45.280642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:35:45.280642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:35:45.281878 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:35:45.281878 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:35:45.281878 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:35:45.281878 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 453/908] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:35:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:35:45.287276 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda9a34a-7dc8-4ac9-ae18-0b09309e7063 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:45.381847 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda9a34a-7dc8-4ac9-ae18-0b09309e7063 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:45.410778 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:35:45.416358 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c30a0-0aa6-4d06-9681-892d6d832463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:45.416955 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c30a0-0aa6-4d06-9681-892d6d832463 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:45.417231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c30a0-0aa6-4d06-9681-892d6d832463 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:45.417501 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c30a0-0aa6-4d06-9681-892d6d832463 None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:45.418124 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c30a0-0aa6-4d06-9681-892d6d832463 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:35:45.418420 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c30a0-0aa6-4d06-9681-892d6d832463 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:45.419795 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c30a0-0aa6-4d06-9681-892d6d832463 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:45.430355 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 456/909] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:35:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:35:45.436079 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0e55c-f440-4e84-bd45-ca190d8b68e5 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:45.489281 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:35:45.535245 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0e55c-f440-4e84-bd45-ca190d8b68e5 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:45.567278 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc575f-612b-4368-a00e-6af70cc09fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:45.567944 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc575f-612b-4368-a00e-6af70cc09fd3 None admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:45.568224 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc575f-612b-4368-a00e-6af70cc09fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:45.568494 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc575f-612b-4368-a00e-6af70cc09fd3 None admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:45.569136 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc575f-612b-4368-a00e-6af70cc09fd3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:45.570541 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc575f-612b-4368-a00e-6af70cc09fd3 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:45.713103 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 454/910] 10.222.0.17 () {60 vars in 1132 bytes} [Mon Oct 17 03:35:45 2022] POST /identity/v3/users => generated 251 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:35:47.857510 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c67c8-fba8-45ea-b950-9595a385c266 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:47.858050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c67c8-fba8-45ea-b950-9595a385c266 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:47.858050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0c67c8-fba8-45ea-b950-9595a385c266 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:47.859405 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 457/911] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:35:47.864763 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ec78c-3bcf-4d47-baca-1bb6bc49fcd3 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:47.865153 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ec78c-3bcf-4d47-baca-1bb6bc49fcd3 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:47.865374 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ec78c-3bcf-4d47-baca-1bb6bc49fcd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:47.883897 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-d62ec78c-3bcf-4d47-baca-1bb6bc49fcd3 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:35:47.888498 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-d62ec78c-3bcf-4d47-baca-1bb6bc49fcd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:35:47.950982 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 455/912] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:35:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:35:48.224972 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c67c8-fba8-45ea-b950-9595a385c266 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:48.325323 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0c67c8-fba8-45ea-b950-9595a385c266 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:48.357385 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b235e-9ea2-4c4c-8ca7-1e4490c3532c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:48.358003 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b235e-9ea2-4c4c-8ca7-1e4490c3532c None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:48.358283 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b235e-9ea2-4c4c-8ca7-1e4490c3532c None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:48.358554 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b235e-9ea2-4c4c-8ca7-1e4490c3532c None admin] PATH_INFO: `/v3/roles/service` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:48.368923 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b235e-9ea2-4c4c-8ca7-1e4490c3532c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:35:48.373664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-531b235e-9ea2-4c4c-8ca7-1e4490c3532c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:35:48.373664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:35:48.373664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:35:48.373664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:35:48.373664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:35:48.373664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:35:48.373664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:35:48.373664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:35:48.373664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:35:48.373664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:35:48.373664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:35:48.373664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:35:48.373664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:35:48.373664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:35:48.373664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:35:48.373664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:35:48.374784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:35:48.374784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:35:48.374784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:35:48.374784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:35:48.374784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:35:48.374784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:35:48.374784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:35:48.374784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:35:48.374784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:35:48.374784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:35:48.374784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:35:48.374784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:35:48.374784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:35:48.374784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:35:48.374784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:35:48.374784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:35:48.375871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:35:48.375871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:35:48.375871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:35:48.375871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:35:48.375871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:35:48.375871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:35:48.375871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:35:48.375871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:35:48.375871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:35:48.375871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:35:48.375871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:35:48.375871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:35:48.375871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 458/913] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Oct 17 03:35:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:35:48.383457 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ec78c-3bcf-4d47-baca-1bb6bc49fcd3 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:48.421949 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:35:48.480567 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ec78c-3bcf-4d47-baca-1bb6bc49fcd3 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:48.514251 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c6468-9f7f-47b9-84b8-8e0e6014946e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:48.514882 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c6468-9f7f-47b9-84b8-8e0e6014946e None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:48.515162 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c6468-9f7f-47b9-84b8-8e0e6014946e None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:48.515434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c6468-9f7f-47b9-84b8-8e0e6014946e None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:48.516134 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c6468-9f7f-47b9-84b8-8e0e6014946e None admin] RBAC: Adding query filter params (name=service) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:35:48.516443 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c6468-9f7f-47b9-84b8-8e0e6014946e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:48.517807 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095c6468-9f7f-47b9-84b8-8e0e6014946e None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:48.525363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 456/914] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Oct 17 03:35:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:35:48.531416 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b235e-9ea2-4c4c-8ca7-1e4490c3532c None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:48.583055 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:35:48.631208 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b235e-9ea2-4c4c-8ca7-1e4490c3532c None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:48.663344 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd8656-deb2-45b3-8dd0-408bf8ccea28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:48.663995 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd8656-deb2-45b3-8dd0-408bf8ccea28 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:48.664272 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd8656-deb2-45b3-8dd0-408bf8ccea28 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:48.664544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd8656-deb2-45b3-8dd0-408bf8ccea28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:48.674708 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbd8656-deb2-45b3-8dd0-408bf8ccea28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:48.676173 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbd8656-deb2-45b3-8dd0-408bf8ccea28 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:48.690342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-afbd8656-deb2-45b3-8dd0-408bf8ccea28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:35:48.690342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:35:48.690342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:35:48.690342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:35:48.690342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:35:48.690342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:35:48.690342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:35:48.690342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:35:48.690342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:35:48.690342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:35:48.690342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:35:48.690342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:35:48.690342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:35:48.690342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:35:48.690342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:35:48.690342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:35:48.691463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:35:48.691463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:35:48.691463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:35:48.691463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:35:48.691463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:35:48.691463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:35:48.691463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:35:48.691463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:35:48.691463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:35:48.691463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:35:48.691463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:35:48.691463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:35:48.691463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:35:48.691463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:35:48.691463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:35:48.691463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:35:48.691463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:35:48.692828 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:35:48.692828 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:35:48.692828 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:35:48.692828 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 459/915] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:35:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 17 03:35:48.697151 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c6468-9f7f-47b9-84b8-8e0e6014946e None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:48.795091 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:35:48.798050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c6468-9f7f-47b9-84b8-8e0e6014946e None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:48.830867 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289722d9-2ea2-4306-97e4-8ab58a4b7466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:48.831471 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289722d9-2ea2-4306-97e4-8ab58a4b7466 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:48.831774 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289722d9-2ea2-4306-97e4-8ab58a4b7466 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:48.832055 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289722d9-2ea2-4306-97e4-8ab58a4b7466 None admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:48.832690 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289722d9-2ea2-4306-97e4-8ab58a4b7466 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:35:48.832994 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289722d9-2ea2-4306-97e4-8ab58a4b7466 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:48.834594 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289722d9-2ea2-4306-97e4-8ab58a4b7466 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:48.845299 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 457/916] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:35:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:35:48.851493 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd8656-deb2-45b3-8dd0-408bf8ccea28 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:48.947119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd8656-deb2-45b3-8dd0-408bf8ccea28 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:48.948602 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:35:48.980768 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6477d2-b88d-4d36-b697-a67322011cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:48.981387 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6477d2-b88d-4d36-b697-a67322011cba None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:48.981661 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6477d2-b88d-4d36-b697-a67322011cba None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:48.981926 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6477d2-b88d-4d36-b697-a67322011cba None admin] PATH_INFO: `/v3/users/placement` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:48.988985 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6477d2-b88d-4d36-b697-a67322011cba None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:48.994823 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6477d2-b88d-4d36-b697-a67322011cba None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:49.001610 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-1e6477d2-b88d-4d36-b697-a67322011cba None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 17 03:35:49.001610 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:35:49.001610 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:35:49.001610 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:35:49.001610 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:35:49.001610 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:35:49.001610 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:35:49.001610 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:35:49.001610 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:35:49.001610 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:35:49.001610 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:35:49.001610 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:35:49.001610 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:35:49.001610 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:35:49.001610 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:35:49.001610 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:35:49.002669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:35:49.002669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:35:49.002669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:35:49.002669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:35:49.002669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:35:49.002669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:35:49.002669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:35:49.002669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:35:49.002669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:35:49.002669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:35:49.002669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:35:49.002669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:35:49.002669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:35:49.002669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:35:49.002669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:35:49.002669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:35:49.002669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:35:49.002669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:35:49.003611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:35:49.003611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:35:49.003611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:35:49.003611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:35:49.003611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:35:49.003611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:35:49.003611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:35:49.003611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:35:49.003611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 17 03:35:49.003611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:35:49.003611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 460/917] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:35:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:35:49.008625 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289722d9-2ea2-4306-97e4-8ab58a4b7466 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:49.109997 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289722d9-2ea2-4306-97e4-8ab58a4b7466 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:49.142993 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd1c9c-6f46-433c-b45a-cfcb56a14aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:49.143643 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd1c9c-6f46-433c-b45a-cfcb56a14aea None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:49.143957 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd1c9c-6f46-433c-b45a-cfcb56a14aea None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:49.144234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd1c9c-6f46-433c-b45a-cfcb56a14aea None admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:49.144956 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd1c9c-6f46-433c-b45a-cfcb56a14aea None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:35:49.145264 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd1c9c-6f46-433c-b45a-cfcb56a14aea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:49.146677 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd1c9c-6f46-433c-b45a-cfcb56a14aea None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:49.164508 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 458/918] 10.222.0.17 () {56 vars in 1143 bytes} [Mon Oct 17 03:35:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:35:49.170371 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6477d2-b88d-4d36-b697-a67322011cba None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:49.268104 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6477d2-b88d-4d36-b697-a67322011cba None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:49.300970 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f1074-735a-4af7-ad9d-6ef210d0bf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:49.301590 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f1074-735a-4af7-ad9d-6ef210d0bf10 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:49.301871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f1074-735a-4af7-ad9d-6ef210d0bf10 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:49.302144 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f1074-735a-4af7-ad9d-6ef210d0bf10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:49.312357 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f1074-735a-4af7-ad9d-6ef210d0bf10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:49.313773 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7f1074-735a-4af7-ad9d-6ef210d0bf10 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:49.327785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-9e7f1074-735a-4af7-ad9d-6ef210d0bf10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:35:49.327785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:35:49.327785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:35:49.327785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:35:49.327785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:35:49.327785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:35:49.327785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:35:49.327785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:35:49.327785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:35:49.327785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:35:49.327785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:35:49.327785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:35:49.327785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:35:49.327785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:35:49.327785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:35:49.327785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:35:49.329855 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:35:49.329855 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:35:49.329855 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:35:49.329855 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:35:49.329855 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:35:49.329855 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:35:49.329855 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:35:49.329855 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:35:49.329855 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:35:49.329855 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:35:49.329855 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:35:49.329855 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:35:49.329855 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:35:49.329855 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:35:49.329855 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:35:49.329855 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:35:49.329855 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:35:49.331525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:35:49.331525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:35:49.331525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:35:49.331525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 461/919] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:35:49.335456 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd1c9c-6f46-433c-b45a-cfcb56a14aea None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:49.354458 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:35:49.433049 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd1c9c-6f46-433c-b45a-cfcb56a14aea None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:49.469303 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f0fd9-827d-4320-9fb7-4322bcbeb28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:49.469918 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f0fd9-827d-4320-9fb7-4322bcbeb28e None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:49.470203 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f0fd9-827d-4320-9fb7-4322bcbeb28e None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:49.470479 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f0fd9-827d-4320-9fb7-4322bcbeb28e None admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:49.471122 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f0fd9-827d-4320-9fb7-4322bcbeb28e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:35:49.471427 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f0fd9-827d-4320-9fb7-4322bcbeb28e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:49.472901 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f0fd9-827d-4320-9fb7-4322bcbeb28e None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:49.484264 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 459/920] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:35:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:35:49.490223 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f1074-735a-4af7-ad9d-6ef210d0bf10 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:49.507031 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:35:49.589514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f1074-735a-4af7-ad9d-6ef210d0bf10 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:49.622188 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413dcd2-3e2c-40f5-a68e-25ea2a3dee26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:49.622808 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413dcd2-3e2c-40f5-a68e-25ea2a3dee26 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:49.623101 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413dcd2-3e2c-40f5-a68e-25ea2a3dee26 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:49.623352 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b413dcd2-3e2c-40f5-a68e-25ea2a3dee26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:49.630106 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b413dcd2-3e2c-40f5-a68e-25ea2a3dee26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:49.631511 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b413dcd2-3e2c-40f5-a68e-25ea2a3dee26 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:49.642278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-b413dcd2-3e2c-40f5-a68e-25ea2a3dee26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:35:49.642278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:35:49.642278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:35:49.642278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:35:49.642278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:35:49.642278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:35:49.642278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:35:49.642278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:35:49.642278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:35:49.642278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:35:49.642278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:35:49.642278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:35:49.642278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:35:49.642278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:35:49.642278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:35:49.644018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:35:49.644018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:35:49.644018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:35:49.644018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:35:49.644018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:35:49.644018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:35:49.644018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:35:49.644018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:35:49.644018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:35:49.644018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:35:49.644018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:35:49.644018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:35:49.644018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:35:49.644018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:35:49.644018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:35:49.644018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:35:49.644018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:35:49.645116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:35:49.645116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:35:49.645116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:35:49.645116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:35:49.645116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:35:49.645116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:35:49.645116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:35:49.645116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:35:49.645116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 462/921] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:35:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:35:49.649844 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f0fd9-827d-4320-9fb7-4322bcbeb28e None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:49.729068 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:35:49.750093 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f0fd9-827d-4320-9fb7-4322bcbeb28e None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:49.782393 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef721155-6fff-4da3-966d-580e026c9955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:49.783030 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef721155-6fff-4da3-966d-580e026c9955 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:49.783305 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef721155-6fff-4da3-966d-580e026c9955 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:49.783575 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef721155-6fff-4da3-966d-580e026c9955 None admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:49.784284 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef721155-6fff-4da3-966d-580e026c9955 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:35:49.784590 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef721155-6fff-4da3-966d-580e026c9955 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:49.785981 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef721155-6fff-4da3-966d-580e026c9955 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:49.798202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 460/922] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Oct 17 03:35:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:35:49.804767 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413dcd2-3e2c-40f5-a68e-25ea2a3dee26 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:49.878515 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:35:49.902110 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b413dcd2-3e2c-40f5-a68e-25ea2a3dee26 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:49.935635 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad47e9a-93bb-4b0e-9c7b-e45885625c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:49.936310 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad47e9a-93bb-4b0e-9c7b-e45885625c2b None admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:49.936588 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad47e9a-93bb-4b0e-9c7b-e45885625c2b None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:49.936858 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad47e9a-93bb-4b0e-9c7b-e45885625c2b None admin] PATH_INFO: `/v3/projects/932cdb9ef7644bf5ba57c6d1d6bce7ff/users/b500a113a2aa4a95875d7e094b917db3/roles/8a0215180fe04d5a8f2c24b370382456` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:49.950960 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad47e9a-93bb-4b0e-9c7b-e45885625c2b None admin] RBAC: Authorizing `identity:create_grant(project_id=932cdb9ef7644bf5ba57c6d1d6bce7ff, user_id=b500a113a2aa4a95875d7e094b917db3, role_id=8a0215180fe04d5a8f2c24b370382456)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:49.952426 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad47e9a-93bb-4b0e-9c7b-e45885625c2b None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:50.018275 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 463/923] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Oct 17 03:35:49 2022] PUT /identity/v3/projects/932cdb9ef7644bf5ba57c6d1d6bce7ff/users/b500a113a2aa4a95875d7e094b917db3/roles/8a0215180fe04d5a8f2c24b370382456 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:35:52.127634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec6483-8f04-4392-aefa-67a0e4a5b927 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:52.128212 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec6483-8f04-4392-aefa-67a0e4a5b927 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:52.128212 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec6483-8f04-4392-aefa-67a0e4a5b927 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:52.129547 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 461/924] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:35:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:35:52.134666 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6479880-d972-400e-9f6e-0e6a60e18b14 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:52.135061 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6479880-d972-400e-9f6e-0e6a60e18b14 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:52.135458 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6479880-d972-400e-9f6e-0e6a60e18b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:52.153748 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-a6479880-d972-400e-9f6e-0e6a60e18b14 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:35:52.155103 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-a6479880-d972-400e-9f6e-0e6a60e18b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:35:52.220508 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 464/925] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:35:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:35:52.484832 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec6483-8f04-4392-aefa-67a0e4a5b927 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:52.586630 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec6483-8f04-4392-aefa-67a0e4a5b927 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:52.619363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2051f0-e9bc-4c5c-8535-b8cd4a9a2b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:52.620015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2051f0-e9bc-4c5c-8535-b8cd4a9a2b90 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:52.620306 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2051f0-e9bc-4c5c-8535-b8cd4a9a2b90 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:52.620577 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2051f0-e9bc-4c5c-8535-b8cd4a9a2b90 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:52.627718 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:35:52.631449 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2051f0-e9bc-4c5c-8535-b8cd4a9a2b90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:35:52.636350 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-ef2051f0-e9bc-4c5c-8535-b8cd4a9a2b90 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:35:52.636350 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:35:52.636350 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:35:52.636350 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:35:52.636350 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:35:52.636350 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:35:52.636350 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:35:52.636350 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:35:52.636350 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:35:52.636350 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:35:52.636350 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:35:52.636350 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:35:52.636350 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:35:52.636350 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:35:52.636350 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:35:52.636350 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:35:52.637434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:35:52.637434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:35:52.637434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:35:52.637434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:35:52.637434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:35:52.637434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:35:52.637434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:35:52.637434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:35:52.637434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:35:52.637434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:35:52.637434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:35:52.637434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:35:52.637434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:35:52.637434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:35:52.637434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:35:52.637434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:35:52.638635 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:35:52.638635 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:35:52.638635 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:35:52.638635 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:35:52.638635 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:35:52.638635 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:35:52.638635 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:35:52.638635 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:35:52.638635 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:35:52.638635 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:35:52.638635 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 17 03:35:52.638635 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:35:52.638635 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 462/926] 10.222.0.17 () {56 vars in 1102 bytes} [Mon Oct 17 03:35:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:35:52.644089 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6479880-d972-400e-9f6e-0e6a60e18b14 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:52.698066 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:35:52.741738 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6479880-d972-400e-9f6e-0e6a60e18b14 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:52.774462 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c047410-17eb-4489-a343-964ca4d0ecb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:52.775063 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c047410-17eb-4489-a343-964ca4d0ecb0 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:52.775348 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c047410-17eb-4489-a343-964ca4d0ecb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:52.775625 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c047410-17eb-4489-a343-964ca4d0ecb0 None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:52.776389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c047410-17eb-4489-a343-964ca4d0ecb0 None admin] RBAC: Adding query filter params (name=service) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:35:52.776692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c047410-17eb-4489-a343-964ca4d0ecb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:52.778031 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c047410-17eb-4489-a343-964ca4d0ecb0 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:52.785819 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 465/927] 10.222.0.17 () {56 vars in 1103 bytes} [Mon Oct 17 03:35:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:35:52.791826 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2051f0-e9bc-4c5c-8535-b8cd4a9a2b90 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:52.891327 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2051f0-e9bc-4c5c-8535-b8cd4a9a2b90 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:52.928059 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee5339-2240-42ba-8e24-9e30e4a01d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:52.928688 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee5339-2240-42ba-8e24-9e30e4a01d85 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:52.928972 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee5339-2240-42ba-8e24-9e30e4a01d85 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:52.929245 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee5339-2240-42ba-8e24-9e30e4a01d85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:52.940034 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee5339-2240-42ba-8e24-9e30e4a01d85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:52.941466 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ee5339-2240-42ba-8e24-9e30e4a01d85 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:52.955800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-28ee5339-2240-42ba-8e24-9e30e4a01d85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:35:52.955800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:35:52.955800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:35:52.955800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:35:52.955800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:35:52.955800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:35:52.955800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:35:52.955800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:35:52.955800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:35:52.955800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:35:52.955800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:35:52.955800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:35:52.955800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:35:52.955800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:35:52.955800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:35:52.955800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:35:52.956950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:35:52.956950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:35:52.956950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:35:52.956950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:35:52.956950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:35:52.956950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:35:52.956950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:35:52.956950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:35:52.956950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:35:52.956950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:35:52.956950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:35:52.956950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:35:52.956950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:35:52.956950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:35:52.956950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:35:52.956950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:35:52.956950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:35:52.958073 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:35:52.958073 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:35:52.958073 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:35:52.958073 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 463/928] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:35:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:35:52.963012 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c047410-17eb-4489-a343-964ca4d0ecb0 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:53.060465 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c047410-17eb-4489-a343-964ca4d0ecb0 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:53.076677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:35:53.093969 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf845d8-219f-448f-a526-b166f563f78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:53.094572 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf845d8-219f-448f-a526-b166f563f78d None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:53.095052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf845d8-219f-448f-a526-b166f563f78d None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:53.095154 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf845d8-219f-448f-a526-b166f563f78d None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:53.096307 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf845d8-219f-448f-a526-b166f563f78d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:35:53.096616 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf845d8-219f-448f-a526-b166f563f78d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:53.097957 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf845d8-219f-448f-a526-b166f563f78d None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:53.110130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 466/929] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:35:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:35:53.116499 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee5339-2240-42ba-8e24-9e30e4a01d85 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:53.165835 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:35:53.213985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee5339-2240-42ba-8e24-9e30e4a01d85 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:53.246836 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6b98e-67e4-4661-a0b8-31b4be68b472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:53.247517 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6b98e-67e4-4661-a0b8-31b4be68b472 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:53.247819 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6b98e-67e4-4661-a0b8-31b4be68b472 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:53.248087 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef6b98e-67e4-4661-a0b8-31b4be68b472 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:53.255233 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef6b98e-67e4-4661-a0b8-31b4be68b472 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:53.256675 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef6b98e-67e4-4661-a0b8-31b4be68b472 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:53.268353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-bef6b98e-67e4-4661-a0b8-31b4be68b472 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 17 03:35:53.268353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:35:53.268353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:35:53.268353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:35:53.268353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:35:53.268353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:35:53.268353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:35:53.268353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:35:53.268353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:35:53.268353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:35:53.268353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:35:53.268353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:35:53.268353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:35:53.268353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:35:53.268353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:35:53.268353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:35:53.271507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:35:53.271507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:35:53.271507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:35:53.271507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:35:53.271507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:35:53.271507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:35:53.271507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:35:53.271507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:35:53.271507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:35:53.271507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:35:53.271507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:35:53.271507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:35:53.271507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:35:53.271507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:35:53.271507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:35:53.271507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:35:53.271507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:35:53.271507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:35:53.272753 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:35:53.272753 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:35:53.272753 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:35:53.272753 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:35:53.272753 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:35:53.272753 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:35:53.272753 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:35:53.272753 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:35:53.272753 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 17 03:35:53.272753 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:35:53.272753 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 464/930] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:35:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:35:53.275406 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf845d8-219f-448f-a526-b166f563f78d None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:53.372335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf845d8-219f-448f-a526-b166f563f78d None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:53.403913 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc98229-0f9c-4a09-bb63-d04dc04db65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:53.404502 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc98229-0f9c-4a09-bb63-d04dc04db65c None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:53.404776 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc98229-0f9c-4a09-bb63-d04dc04db65c None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:53.405041 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc98229-0f9c-4a09-bb63-d04dc04db65c None admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:53.405746 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc98229-0f9c-4a09-bb63-d04dc04db65c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:35:53.406043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc98229-0f9c-4a09-bb63-d04dc04db65c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:53.407424 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc98229-0f9c-4a09-bb63-d04dc04db65c None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:53.424852 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 467/931] 10.222.0.17 () {56 vars in 1143 bytes} [Mon Oct 17 03:35:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:35:53.430539 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6b98e-67e4-4661-a0b8-31b4be68b472 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:53.529040 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef6b98e-67e4-4661-a0b8-31b4be68b472 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:53.539759 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:35:53.562522 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740baef4-235a-4f62-83a5-d29431358140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:53.563128 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740baef4-235a-4f62-83a5-d29431358140 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:53.563413 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740baef4-235a-4f62-83a5-d29431358140 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:53.563690 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740baef4-235a-4f62-83a5-d29431358140 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:53.629581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740baef4-235a-4f62-83a5-d29431358140 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:53.631003 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740baef4-235a-4f62-83a5-d29431358140 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:53.728349 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 465/932] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:35:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 03:35:53.734099 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc98229-0f9c-4a09-bb63-d04dc04db65c None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:53.766483 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:35:53.829427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc98229-0f9c-4a09-bb63-d04dc04db65c None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:53.862091 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1ccda-393e-4480-bdc7-048f7101080e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:53.862702 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1ccda-393e-4480-bdc7-048f7101080e None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:53.862983 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1ccda-393e-4480-bdc7-048f7101080e None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:53.863250 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1ccda-393e-4480-bdc7-048f7101080e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:53.872812 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b1ccda-393e-4480-bdc7-048f7101080e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:53.874232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b1ccda-393e-4480-bdc7-048f7101080e None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:53.888721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-d6b1ccda-393e-4480-bdc7-048f7101080e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:35:53.888721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:35:53.888721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:35:53.888721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:35:53.888721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:35:53.888721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:35:53.888721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:35:53.888721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:35:53.888721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:35:53.888721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:35:53.888721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:35:53.888721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:35:53.888721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:35:53.888721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:35:53.888721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:35:53.888721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:35:53.889806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:35:53.889806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:35:53.889806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:35:53.889806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:35:53.889806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:35:53.889806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:35:53.889806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:35:53.889806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:35:53.889806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:35:53.889806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:35:53.889806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:35:53.889806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:35:53.889806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:35:53.889806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:35:53.889806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:35:53.889806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:35:53.889806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:35:53.891097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:35:53.891097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:35:53.891097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:35:53.891097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 468/933] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:35:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:35:53.896131 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740baef4-235a-4f62-83a5-d29431358140 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:53.907671 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:35:53.992444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740baef4-235a-4f62-83a5-d29431358140 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:54.026726 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c4c698-401f-4178-8167-5844d5999d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:54.027320 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4c698-401f-4178-8167-5844d5999d81 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:54.027596 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4c698-401f-4178-8167-5844d5999d81 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:54.027896 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4c698-401f-4178-8167-5844d5999d81 None admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:54.028527 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c4c698-401f-4178-8167-5844d5999d81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:35:54.028827 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c4c698-401f-4178-8167-5844d5999d81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:54.030203 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c4c698-401f-4178-8167-5844d5999d81 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:54.041336 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 466/934] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:35:54.047229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1ccda-393e-4480-bdc7-048f7101080e None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:54.131454 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:35:54.146381 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1ccda-393e-4480-bdc7-048f7101080e None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:54.178072 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b976e2d-cf9e-42f8-a32b-1ca02c56447f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:54.178686 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b976e2d-cf9e-42f8-a32b-1ca02c56447f None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:54.178967 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b976e2d-cf9e-42f8-a32b-1ca02c56447f None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:54.179242 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b976e2d-cf9e-42f8-a32b-1ca02c56447f None admin] PATH_INFO: `/v3/projects/service` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:54.185604 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b976e2d-cf9e-42f8-a32b-1ca02c56447f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:54.187021 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b976e2d-cf9e-42f8-a32b-1ca02c56447f None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:54.198466 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-7b976e2d-cf9e-42f8-a32b-1ca02c56447f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:35:54.198466 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:35:54.198466 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:35:54.198466 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:35:54.198466 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:35:54.198466 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:35:54.198466 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:35:54.198466 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:35:54.198466 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:35:54.198466 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:35:54.198466 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:35:54.198466 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:35:54.198466 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:35:54.198466 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:35:54.198466 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:35:54.199514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:35:54.199514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:35:54.199514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:35:54.199514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:35:54.199514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:35:54.199514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:35:54.199514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:35:54.199514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:35:54.199514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:35:54.199514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:35:54.199514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:35:54.199514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:35:54.199514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:35:54.199514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:35:54.199514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:35:54.199514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:35:54.199514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:35:54.200779 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:35:54.200779 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:35:54.200779 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:35:54.200779 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:35:54.200779 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:35:54.200779 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:35:54.200779 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:35:54.200779 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:35:54.200779 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 469/935] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:35:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:35:54.206166 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c4c698-401f-4178-8167-5844d5999d81 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:54.274156 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:35:54.303809 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c4c698-401f-4178-8167-5844d5999d81 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:54.336949 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc456c7-4611-40d9-8ee8-14c03b36e83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:54.337734 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc456c7-4611-40d9-8ee8-14c03b36e83b None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:54.338441 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc456c7-4611-40d9-8ee8-14c03b36e83b None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:54.338712 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc456c7-4611-40d9-8ee8-14c03b36e83b None admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:54.339393 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc456c7-4611-40d9-8ee8-14c03b36e83b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:35:54.339700 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc456c7-4611-40d9-8ee8-14c03b36e83b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:54.341180 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc456c7-4611-40d9-8ee8-14c03b36e83b None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:54.353303 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 467/936] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Oct 17 03:35:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:35:54.359404 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b976e2d-cf9e-42f8-a32b-1ca02c56447f None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:54.457472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b976e2d-cf9e-42f8-a32b-1ca02c56447f None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:54.489817 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0e2a52-dcf7-4aad-aa6a-11534772c66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:54.490391 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e2a52-dcf7-4aad-aa6a-11534772c66a None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:54.490667 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e2a52-dcf7-4aad-aa6a-11534772c66a None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:54.490935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0e2a52-dcf7-4aad-aa6a-11534772c66a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:54.491652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0e2a52-dcf7-4aad-aa6a-11534772c66a None admin] RBAC: Adding query filter params (role.id=8a0215180fe04d5a8f2c24b370382456, scope.project.id=932cdb9ef7644bf5ba57c6d1d6bce7ff, user.id=b500a113a2aa4a95875d7e094b917db3) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:35:54.491983 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0e2a52-dcf7-4aad-aa6a-11534772c66a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:54.493412 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0e2a52-dcf7-4aad-aa6a-11534772c66a None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:54.498901 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 470/937] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Oct 17 03:35:54 2022] GET /identity/v3/role_assignments?user.id=b500a113a2aa4a95875d7e094b917db3&scope.project.id=932cdb9ef7644bf5ba57c6d1d6bce7ff&role.id=8a0215180fe04d5a8f2c24b370382456 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:35:56.657454 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750eef0-c85f-4f3c-99d7-61d216e53b07 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:56.658670 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750eef0-c85f-4f3c-99d7-61d216e53b07 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:56.658670 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2750eef0-c85f-4f3c-99d7-61d216e53b07 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:56.659890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 468/938] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:35:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:35:56.665355 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766518f2-f744-4955-a543-3c0acc837cb2 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:56.667146 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766518f2-f744-4955-a543-3c0acc837cb2 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:56.667146 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766518f2-f744-4955-a543-3c0acc837cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:56.683963 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-766518f2-f744-4955-a543-3c0acc837cb2 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:35:56.685249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-766518f2-f744-4955-a543-3c0acc837cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:35:56.688203 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:35:56.751254 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 471/939] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:35:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:35:57.015299 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750eef0-c85f-4f3c-99d7-61d216e53b07 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:57.115880 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2750eef0-c85f-4f3c-99d7-61d216e53b07 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:57.145651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:35:57.151309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753ab56-254a-484c-b171-468f1d35abd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:57.151975 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753ab56-254a-484c-b171-468f1d35abd9 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:57.152259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753ab56-254a-484c-b171-468f1d35abd9 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:57.152534 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a753ab56-254a-484c-b171-468f1d35abd9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:57.162238 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a753ab56-254a-484c-b171-468f1d35abd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:35:57.167129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-a753ab56-254a-484c-b171-468f1d35abd9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:35:57.167129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:35:57.167129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:35:57.167129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:35:57.167129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:35:57.167129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:35:57.167129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:35:57.167129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:35:57.167129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:35:57.167129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:35:57.167129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:35:57.167129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:35:57.167129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:35:57.167129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:35:57.167129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:35:57.167129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:35:57.168300 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:35:57.168300 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:35:57.168300 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:35:57.168300 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:35:57.168300 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:35:57.168300 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:35:57.168300 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:35:57.168300 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:35:57.168300 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:35:57.168300 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:35:57.168300 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:35:57.168300 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:35:57.168300 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:35:57.168300 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:35:57.168300 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:35:57.168300 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:35:57.169382 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:35:57.169382 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:35:57.169382 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:35:57.169382 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:35:57.169382 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:35:57.169382 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:35:57.169382 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:35:57.169382 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:35:57.169382 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:35:57.169382 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:35:57.169382 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:35:57.169382 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:35:57.169382 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 469/940] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Oct 17 03:35:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:35:57.174367 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766518f2-f744-4955-a543-3c0acc837cb2 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:57.270110 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766518f2-f744-4955-a543-3c0acc837cb2 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:57.303881 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcddf8c-a5a5-4203-99aa-6a4c5c1db839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:57.304477 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcddf8c-a5a5-4203-99aa-6a4c5c1db839 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:57.304754 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcddf8c-a5a5-4203-99aa-6a4c5c1db839 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:57.305021 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcddf8c-a5a5-4203-99aa-6a4c5c1db839 None admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:57.305669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcddf8c-a5a5-4203-99aa-6a4c5c1db839 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:35:57.305967 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcddf8c-a5a5-4203-99aa-6a4c5c1db839 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:57.307293 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcddf8c-a5a5-4203-99aa-6a4c5c1db839 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:57.315669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 472/941] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:35:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:35:57.321656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753ab56-254a-484c-b171-468f1d35abd9 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:57.420783 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a753ab56-254a-484c-b171-468f1d35abd9 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:57.454633 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027292f-b642-4892-a4ea-3d5b0f757346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:57.455250 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027292f-b642-4892-a4ea-3d5b0f757346 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:57.455531 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027292f-b642-4892-a4ea-3d5b0f757346 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:57.455837 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027292f-b642-4892-a4ea-3d5b0f757346 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:57.465726 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d027292f-b642-4892-a4ea-3d5b0f757346 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:57.467152 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d027292f-b642-4892-a4ea-3d5b0f757346 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:57.481616 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-d027292f-b642-4892-a4ea-3d5b0f757346 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:35:57.481616 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:35:57.481616 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:35:57.481616 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:35:57.481616 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:35:57.481616 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:35:57.481616 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:35:57.481616 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:35:57.481616 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:35:57.481616 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:35:57.481616 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:35:57.481616 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:35:57.481616 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:35:57.481616 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:35:57.481616 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:35:57.481616 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:35:57.482754 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:35:57.482754 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:35:57.482754 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:35:57.482754 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:35:57.482754 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:35:57.482754 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:35:57.482754 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:35:57.482754 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:35:57.482754 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:35:57.482754 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:35:57.482754 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:35:57.482754 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:35:57.482754 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:35:57.482754 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:35:57.482754 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:35:57.482754 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:35:57.482754 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:35:57.483902 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:35:57.483902 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:35:57.483902 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:35:57.483902 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 470/942] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:35:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:35:57.490981 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcddf8c-a5a5-4203-99aa-6a4c5c1db839 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:57.493920 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:35:57.588709 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcddf8c-a5a5-4203-99aa-6a4c5c1db839 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:57.621696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ccb09-e0ae-4a21-8f32-4be70c4058d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:57.624117 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ccb09-e0ae-4a21-8f32-4be70c4058d3 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:57.624117 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ccb09-e0ae-4a21-8f32-4be70c4058d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:57.624117 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ccb09-e0ae-4a21-8f32-4be70c4058d3 None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:57.624117 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ccb09-e0ae-4a21-8f32-4be70c4058d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:35:57.624117 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ccb09-e0ae-4a21-8f32-4be70c4058d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:57.625346 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ccb09-e0ae-4a21-8f32-4be70c4058d3 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:57.637178 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 473/943] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:35:57.643005 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027292f-b642-4892-a4ea-3d5b0f757346 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:57.686051 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:35:57.742184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027292f-b642-4892-a4ea-3d5b0f757346 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:57.775326 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c9b32-bee3-482b-956f-909963f9902c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:57.775982 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c9b32-bee3-482b-956f-909963f9902c None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:57.776265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c9b32-bee3-482b-956f-909963f9902c None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:57.776538 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c9b32-bee3-482b-956f-909963f9902c None admin] PATH_INFO: `/v3/users/placement` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:57.784255 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c9b32-bee3-482b-956f-909963f9902c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:57.785666 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c9b32-bee3-482b-956f-909963f9902c None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:57.796599 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-922c9b32-bee3-482b-956f-909963f9902c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 17 03:35:57.796599 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:35:57.796599 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:35:57.796599 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:35:57.796599 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:35:57.796599 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:35:57.796599 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:35:57.796599 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:35:57.796599 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:35:57.796599 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:35:57.796599 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:35:57.796599 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:35:57.796599 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:35:57.796599 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:35:57.796599 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:35:57.796599 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:35:57.803206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:35:57.803206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:35:57.803206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:35:57.803206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:35:57.803206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:35:57.803206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:35:57.803206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:35:57.803206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:35:57.803206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:35:57.803206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:35:57.803206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:35:57.803206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:35:57.803206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:35:57.803206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:35:57.803206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:35:57.803206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:35:57.803206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:35:57.803206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:35:57.804412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:35:57.804412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:35:57.804412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:35:57.804412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:35:57.804412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:35:57.804412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:35:57.804412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:35:57.804412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:35:57.804412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 17 03:35:57.804412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:35:57.804412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 471/944] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:35:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:35:57.805155 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ccb09-e0ae-4a21-8f32-4be70c4058d3 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:57.865162 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:35:57.900351 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ccb09-e0ae-4a21-8f32-4be70c4058d3 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:57.933696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36d7a2-0b8a-45d6-841e-5ece4c91cc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:57.934297 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36d7a2-0b8a-45d6-841e-5ece4c91cc9c None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:57.934579 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36d7a2-0b8a-45d6-841e-5ece4c91cc9c None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:57.934853 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36d7a2-0b8a-45d6-841e-5ece4c91cc9c None admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:57.935654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36d7a2-0b8a-45d6-841e-5ece4c91cc9c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:35:57.936055 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36d7a2-0b8a-45d6-841e-5ece4c91cc9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:57.937459 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36d7a2-0b8a-45d6-841e-5ece4c91cc9c None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:57.954253 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 474/945] 10.222.0.17 () {56 vars in 1143 bytes} [Mon Oct 17 03:35:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:35:57.960906 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c9b32-bee3-482b-956f-909963f9902c None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:58.058494 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c9b32-bee3-482b-956f-909963f9902c None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:58.059986 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:35:58.094277 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc97fb2-b65c-4e5f-b9a6-3e98ade3779a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:58.094893 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc97fb2-b65c-4e5f-b9a6-3e98ade3779a None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:58.095227 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc97fb2-b65c-4e5f-b9a6-3e98ade3779a None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:58.096008 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc97fb2-b65c-4e5f-b9a6-3e98ade3779a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:58.105520 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc97fb2-b65c-4e5f-b9a6-3e98ade3779a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:58.106950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc97fb2-b65c-4e5f-b9a6-3e98ade3779a None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:58.121587 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-abc97fb2-b65c-4e5f-b9a6-3e98ade3779a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:35:58.121587 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:35:58.121587 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:35:58.121587 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:35:58.121587 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:35:58.121587 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:35:58.121587 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:35:58.121587 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:35:58.121587 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:35:58.121587 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:35:58.121587 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:35:58.121587 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:35:58.121587 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:35:58.121587 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:35:58.121587 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:35:58.121587 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:35:58.122686 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:35:58.122686 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:35:58.122686 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:35:58.122686 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:35:58.122686 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:35:58.122686 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:35:58.122686 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:35:58.122686 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:35:58.122686 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:35:58.122686 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:35:58.122686 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:35:58.122686 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:35:58.122686 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:35:58.122686 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:35:58.122686 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:35:58.122686 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:35:58.122686 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:35:58.123818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:35:58.123818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:35:58.123818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:35:58.123818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 472/946] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:35:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:35:58.128597 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36d7a2-0b8a-45d6-841e-5ece4c91cc9c None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:58.225465 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36d7a2-0b8a-45d6-841e-5ece4c91cc9c None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:58.245042 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:35:58.258282 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd07cb3-2961-4dd6-8e33-5461e0874ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:58.258879 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd07cb3-2961-4dd6-8e33-5461e0874ea3 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:58.259157 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd07cb3-2961-4dd6-8e33-5461e0874ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:58.259428 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd07cb3-2961-4dd6-8e33-5461e0874ea3 None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:58.260097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd07cb3-2961-4dd6-8e33-5461e0874ea3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:35:58.260399 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd07cb3-2961-4dd6-8e33-5461e0874ea3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:58.261737 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd07cb3-2961-4dd6-8e33-5461e0874ea3 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:58.272898 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 475/947] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:35:58.278468 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc97fb2-b65c-4e5f-b9a6-3e98ade3779a None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:58.377769 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc97fb2-b65c-4e5f-b9a6-3e98ade3779a None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:58.411817 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0400a1e-175a-4d54-9a94-c50f95d95d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:58.412437 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0400a1e-175a-4d54-9a94-c50f95d95d4c None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:58.412715 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0400a1e-175a-4d54-9a94-c50f95d95d4c None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:58.412986 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0400a1e-175a-4d54-9a94-c50f95d95d4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:58.419602 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0400a1e-175a-4d54-9a94-c50f95d95d4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:58.421065 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0400a1e-175a-4d54-9a94-c50f95d95d4c None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:58.432097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-a0400a1e-175a-4d54-9a94-c50f95d95d4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:35:58.432097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:35:58.432097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:35:58.432097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:35:58.432097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:35:58.432097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:35:58.432097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:35:58.432097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:35:58.432097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:35:58.432097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:35:58.432097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:35:58.432097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:35:58.432097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:35:58.432097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:35:58.432097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:35:58.433185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:35:58.433185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:35:58.433185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:35:58.433185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:35:58.433185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:35:58.433185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:35:58.433185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:35:58.433185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:35:58.433185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:35:58.433185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:35:58.433185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:35:58.433185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:35:58.433185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:35:58.433185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:35:58.433185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:35:58.433185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:35:58.433185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:35:58.434433 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:35:58.434433 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:35:58.434433 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:35:58.434433 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:35:58.434433 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:35:58.434433 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:35:58.434433 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:35:58.434433 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:35:58.434433 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 473/948] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:35:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:35:58.439584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd07cb3-2961-4dd6-8e33-5461e0874ea3 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:58.536324 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd07cb3-2961-4dd6-8e33-5461e0874ea3 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:58.568650 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c63819-1ca3-43cd-a5fd-f183f7384775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:58.569244 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c63819-1ca3-43cd-a5fd-f183f7384775 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:58.569520 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c63819-1ca3-43cd-a5fd-f183f7384775 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:58.569787 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c63819-1ca3-43cd-a5fd-f183f7384775 None admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:58.570465 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c63819-1ca3-43cd-a5fd-f183f7384775 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:35:58.570767 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c63819-1ca3-43cd-a5fd-f183f7384775 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:58.572202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c63819-1ca3-43cd-a5fd-f183f7384775 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:58.584362 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 476/949] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Oct 17 03:35:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:35:58.590440 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0400a1e-175a-4d54-9a94-c50f95d95d4c None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:35:58.606724 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:35:58.687848 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0400a1e-175a-4d54-9a94-c50f95d95d4c None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:35:58.723652 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0bc11-b3ca-4e5e-b62f-84336fc2dfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:35:58.724327 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0bc11-b3ca-4e5e-b62f-84336fc2dfd4 None admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:35:58.724610 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0bc11-b3ca-4e5e-b62f-84336fc2dfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:35:58.724884 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0bc11-b3ca-4e5e-b62f-84336fc2dfd4 None admin] PATH_INFO: `/v3/projects/932cdb9ef7644bf5ba57c6d1d6bce7ff/users/b500a113a2aa4a95875d7e094b917db3/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:35:58.726884 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda0bc11-b3ca-4e5e-b62f-84336fc2dfd4 None admin] RBAC: Authorizing `identity:create_grant(project_id=932cdb9ef7644bf5ba57c6d1d6bce7ff, user_id=b500a113a2aa4a95875d7e094b917db3, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:35:58.728343 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda0bc11-b3ca-4e5e-b62f-84336fc2dfd4 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:35:58.865502 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 474/950] 10.222.0.17 () {58 vars in 1439 bytes} [Mon Oct 17 03:35:58 2022] PUT /identity/v3/projects/932cdb9ef7644bf5ba57c6d1d6bce7ff/users/b500a113a2aa4a95875d7e094b917db3/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:36:00.992442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6a2d8-c4ae-4bcb-a0c0-1e99d3da4fe3 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:00.992968 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6a2d8-c4ae-4bcb-a0c0-1e99d3da4fe3 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:00.992968 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6a2d8-c4ae-4bcb-a0c0-1e99d3da4fe3 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:00.994464 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 477/951] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:36:00.999836 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719dcc6-cbd9-4d12-9ae6-aa60393f2be6 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:01.000271 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719dcc6-cbd9-4d12-9ae6-aa60393f2be6 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:01.000425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8719dcc6-cbd9-4d12-9ae6-aa60393f2be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:01.018801 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-8719dcc6-cbd9-4d12-9ae6-aa60393f2be6 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:36:01.020020 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-8719dcc6-cbd9-4d12-9ae6-aa60393f2be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:36:01.084896 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 475/952] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:36:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:36:01.354610 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e6a2d8-c4ae-4bcb-a0c0-1e99d3da4fe3 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:36:01.396950 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:36:01.453252 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e6a2d8-c4ae-4bcb-a0c0-1e99d3da4fe3 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:36:01.486095 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00359cfd-28d1-4603-96a9-05afeebf2a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:36:01.486720 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00359cfd-28d1-4603-96a9-05afeebf2a3b None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:01.486999 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00359cfd-28d1-4603-96a9-05afeebf2a3b None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:01.487270 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00359cfd-28d1-4603-96a9-05afeebf2a3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:01.497970 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00359cfd-28d1-4603-96a9-05afeebf2a3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 17 03:36:01.502785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-00359cfd-28d1-4603-96a9-05afeebf2a3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:36:01.502785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:36:01.502785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:36:01.502785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:36:01.502785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:36:01.502785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:36:01.502785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:36:01.502785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:36:01.502785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:36:01.502785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:36:01.502785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:36:01.502785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:36:01.502785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 17 03:36:01.502785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 17 03:36:01.502785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 17 03:36:01.502785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 17 03:36:01.504114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 17 03:36:01.504114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 17 03:36:01.504114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 17 03:36:01.504114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 17 03:36:01.504114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:36:01.504114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:36:01.504114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:36:01.504114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:36:01.504114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:36:01.504114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:36:01.504114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:36:01.504114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:36:01.504114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:36:01.504114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:36:01.504114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:36:01.504114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:36:01.504965 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:36:01.504965 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:36:01.504965 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:36:01.504965 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:36:01.504965 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 17 03:36:01.504965 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 17 03:36:01.504965 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 17 03:36:01.504965 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 17 03:36:01.504965 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 17 03:36:01.504965 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 17 03:36:01.504965 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 17 03:36:01.504965 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:36:01.504965 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 478/953] 10.222.0.17 () {56 vars in 1096 bytes} [Mon Oct 17 03:36:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:36:01.510045 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719dcc6-cbd9-4d12-9ae6-aa60393f2be6 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:36:01.513186 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:36:01.607557 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8719dcc6-cbd9-4d12-9ae6-aa60393f2be6 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:36:01.642585 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3df65a-a70a-4a8a-807b-69d70c366197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:36:01.643196 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3df65a-a70a-4a8a-807b-69d70c366197 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:01.643482 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3df65a-a70a-4a8a-807b-69d70c366197 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:01.643772 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3df65a-a70a-4a8a-807b-69d70c366197 None admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:01.644438 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3df65a-a70a-4a8a-807b-69d70c366197 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:36:01.644738 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3df65a-a70a-4a8a-807b-69d70c366197 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:36:01.646094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3df65a-a70a-4a8a-807b-69d70c366197 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:36:01.654324 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 476/954] 10.222.0.17 () {56 vars in 1099 bytes} [Mon Oct 17 03:36:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:36:01.660754 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00359cfd-28d1-4603-96a9-05afeebf2a3b None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:36:01.757424 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:36:01.760346 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00359cfd-28d1-4603-96a9-05afeebf2a3b None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:36:01.793865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3df6f-b380-49df-ae7f-fa8c31530c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:36:01.794486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3df6f-b380-49df-ae7f-fa8c31530c9a None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:01.794767 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3df6f-b380-49df-ae7f-fa8c31530c9a None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:01.795037 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3df6f-b380-49df-ae7f-fa8c31530c9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:01.804590 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3df6f-b380-49df-ae7f-fa8c31530c9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:36:01.809678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3df6f-b380-49df-ae7f-fa8c31530c9a None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:36:01.819899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-5cd3df6f-b380-49df-ae7f-fa8c31530c9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:36:01.819899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:36:01.819899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:36:01.819899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:36:01.819899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:36:01.819899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:36:01.819899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:36:01.819899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:36:01.819899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:36:01.819899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:36:01.819899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:36:01.819899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:36:01.819899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:36:01.819899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:36:01.819899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:36:01.819899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:36:01.821024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:36:01.821024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:36:01.821024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:36:01.821024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:36:01.821024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:36:01.821024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:36:01.821024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:36:01.821024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:36:01.821024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:36:01.821024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:36:01.821024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:36:01.821024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:36:01.821024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:36:01.821024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:36:01.821024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:36:01.821024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:36:01.821024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:36:01.822268 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:36:01.822268 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:36:01.822268 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:36:01.822268 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 479/955] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:36:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:36:01.826849 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3df65a-a70a-4a8a-807b-69d70c366197 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:36:01.891056 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:36:01.924954 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3df65a-a70a-4a8a-807b-69d70c366197 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:36:01.956746 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287bf6c9-5d5c-4090-a483-725d797700ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:36:01.957333 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bf6c9-5d5c-4090-a483-725d797700ef None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:01.957613 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bf6c9-5d5c-4090-a483-725d797700ef None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:01.957881 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bf6c9-5d5c-4090-a483-725d797700ef None admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:01.958510 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287bf6c9-5d5c-4090-a483-725d797700ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:36:01.959074 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287bf6c9-5d5c-4090-a483-725d797700ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:36:01.960446 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287bf6c9-5d5c-4090-a483-725d797700ef None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:36:01.972722 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 477/956] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:36:01.978639 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3df6f-b380-49df-ae7f-fa8c31530c9a None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:36:02.075132 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3df6f-b380-49df-ae7f-fa8c31530c9a None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:36:02.108750 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0de5e9-71ae-4b2a-999a-4ad48d313c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:36:02.109380 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0de5e9-71ae-4b2a-999a-4ad48d313c91 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:02.109657 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0de5e9-71ae-4b2a-999a-4ad48d313c91 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:02.109924 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0de5e9-71ae-4b2a-999a-4ad48d313c91 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:02.118117 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0de5e9-71ae-4b2a-999a-4ad48d313c91 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:36:02.119536 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0de5e9-71ae-4b2a-999a-4ad48d313c91 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:36:02.130990 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-5f0de5e9-71ae-4b2a-999a-4ad48d313c91 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 17 03:36:02.130990 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:36:02.130990 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:36:02.130990 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:36:02.130990 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:36:02.130990 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:36:02.130990 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:36:02.130990 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:36:02.130990 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:36:02.130990 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:36:02.130990 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:36:02.130990 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:36:02.130990 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 17 03:36:02.130990 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 17 03:36:02.130990 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 17 03:36:02.130990 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 17 03:36:02.132153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:36:02.132153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:36:02.132153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 17 03:36:02.132153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:36:02.132153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 17 03:36:02.132153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 17 03:36:02.132153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:36:02.132153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:36:02.132153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:36:02.132153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:36:02.132153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:36:02.132153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:36:02.132153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:36:02.132153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:36:02.132153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:36:02.132153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:36:02.132153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:36:02.132153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:36:02.133088 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:36:02.133088 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:36:02.133088 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 17 03:36:02.133088 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 17 03:36:02.133088 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 17 03:36:02.133088 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 17 03:36:02.133088 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 17 03:36:02.133088 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 17 03:36:02.133088 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 17 03:36:02.133088 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:36:02.133088 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 480/957] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:36:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:36:02.137887 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287bf6c9-5d5c-4090-a483-725d797700ef None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:36:02.236242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287bf6c9-5d5c-4090-a483-725d797700ef None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:36:02.261576 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:36:02.270566 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b92b6f-28aa-4617-b24c-52926e05a32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:36:02.271173 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b92b6f-28aa-4617-b24c-52926e05a32e None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:02.271457 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b92b6f-28aa-4617-b24c-52926e05a32e None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:02.271760 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b92b6f-28aa-4617-b24c-52926e05a32e None admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:02.272485 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b92b6f-28aa-4617-b24c-52926e05a32e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:36:02.272793 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b92b6f-28aa-4617-b24c-52926e05a32e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:36:02.274206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b92b6f-28aa-4617-b24c-52926e05a32e None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:36:02.290637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 478/958] 10.222.0.17 () {56 vars in 1143 bytes} [Mon Oct 17 03:36:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:36:02.296625 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0de5e9-71ae-4b2a-999a-4ad48d313c91 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:36:02.311348 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:36:02.392745 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0de5e9-71ae-4b2a-999a-4ad48d313c91 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:36:02.424448 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded840b-238c-43d2-85c9-5f94d905f8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:36:02.425172 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded840b-238c-43d2-85c9-5f94d905f8db None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:02.425460 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded840b-238c-43d2-85c9-5f94d905f8db None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:02.425730 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded840b-238c-43d2-85c9-5f94d905f8db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:02.490056 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded840b-238c-43d2-85c9-5f94d905f8db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:36:02.491425 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded840b-238c-43d2-85c9-5f94d905f8db None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:36:02.504972 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:36:02.586336 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 481/959] 10.222.0.17 () {58 vars in 1282 bytes} [Mon Oct 17 03:36:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 17 03:36:02.592170 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b92b6f-28aa-4617-b24c-52926e05a32e None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:36:02.689370 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b92b6f-28aa-4617-b24c-52926e05a32e None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:36:02.723164 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f33e83-1a8b-4354-b05f-cbf54e0a5122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:36:02.723811 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f33e83-1a8b-4354-b05f-cbf54e0a5122 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:02.724092 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f33e83-1a8b-4354-b05f-cbf54e0a5122 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:02.724360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f33e83-1a8b-4354-b05f-cbf54e0a5122 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:02.733964 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f33e83-1a8b-4354-b05f-cbf54e0a5122 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:36:02.735903 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f33e83-1a8b-4354-b05f-cbf54e0a5122 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:36:02.749904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-d2f33e83-1a8b-4354-b05f-cbf54e0a5122 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:36:02.749904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:36:02.749904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:36:02.749904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:36:02.749904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:36:02.749904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:36:02.749904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:36:02.749904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:36:02.749904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:36:02.749904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:36:02.749904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:36:02.749904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:36:02.749904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 17 03:36:02.749904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 17 03:36:02.749904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 17 03:36:02.749904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 17 03:36:02.750880 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:36:02.750880 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:36:02.750880 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:36:02.750880 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:36:02.750880 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:36:02.750880 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:36:02.750880 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:36:02.750880 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:36:02.750880 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:36:02.750880 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:36:02.750880 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:36:02.750880 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:36:02.750880 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:36:02.750880 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:36:02.750880 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:36:02.750880 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:36:02.750880 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 17 03:36:02.752439 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 17 03:36:02.752439 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 17 03:36:02.752439 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:36:02.752439 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 479/960] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:36:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:36:02.757347 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded840b-238c-43d2-85c9-5f94d905f8db None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:36:02.854688 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded840b-238c-43d2-85c9-5f94d905f8db None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:36:02.867797 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:36:02.887327 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cbb351-178b-48a4-aa8c-9ef6794f273b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:36:02.887947 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbb351-178b-48a4-aa8c-9ef6794f273b None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:02.888227 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbb351-178b-48a4-aa8c-9ef6794f273b None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:02.888496 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbb351-178b-48a4-aa8c-9ef6794f273b None admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:02.889118 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cbb351-178b-48a4-aa8c-9ef6794f273b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:36:02.889415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cbb351-178b-48a4-aa8c-9ef6794f273b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:36:02.890748 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cbb351-178b-48a4-aa8c-9ef6794f273b None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:36:02.901652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 482/961] 10.222.0.17 () {56 vars in 1109 bytes} [Mon Oct 17 03:36:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:36:02.907696 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f33e83-1a8b-4354-b05f-cbf54e0a5122 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:36:02.943050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:36:03.005166 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f33e83-1a8b-4354-b05f-cbf54e0a5122 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:36:03.038920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18263734-0789-4ad4-ac60-ce5cb3fbd5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:36:03.039539 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18263734-0789-4ad4-ac60-ce5cb3fbd5e9 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:03.039848 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18263734-0789-4ad4-ac60-ce5cb3fbd5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:03.040124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18263734-0789-4ad4-ac60-ce5cb3fbd5e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:03.046547 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18263734-0789-4ad4-ac60-ce5cb3fbd5e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:36:03.048750 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18263734-0789-4ad4-ac60-ce5cb3fbd5e9 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:36:03.060650 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application [None req-18263734-0789-4ad4-ac60-ce5cb3fbd5e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:36:03.060650 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:36:03.060650 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:36:03.060650 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:36:03.060650 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:36:03.060650 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:36:03.060650 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:36:03.060650 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:36:03.060650 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:36:03.060650 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:36:03.060650 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:36:03.060650 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:36:03.060650 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 17 03:36:03.060650 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 17 03:36:03.060650 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 17 03:36:03.061741 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 17 03:36:03.061741 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:36:03.061741 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:36:03.061741 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:36:03.061741 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:36:03.061741 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:36:03.061741 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:36:03.061741 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:36:03.061741 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application with Lock( Oct 17 03:36:03.061741 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:36:03.061741 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:36:03.061741 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:36:03.061741 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:36:03.061741 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:36:03.061741 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:36:03.061741 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:36:03.061741 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:36:03.062844 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 17 03:36:03.062844 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 17 03:36:03.062844 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 17 03:36:03.062844 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 17 03:36:03.062844 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 17 03:36:03.062844 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 17 03:36:03.062844 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 17 03:36:03.062844 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: ERROR keystone.server.flask.application Oct 17 03:36:03.062844 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 480/962] 10.222.0.17 () {56 vars in 1111 bytes} [Mon Oct 17 03:36:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:36:03.067666 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cbb351-178b-48a4-aa8c-9ef6794f273b None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:36:03.162126 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cbb351-178b-48a4-aa8c-9ef6794f273b None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:36:03.194978 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9349245f-61cc-480d-b932-8e1248470f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:36:03.195575 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349245f-61cc-480d-b932-8e1248470f71 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:03.195887 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349245f-61cc-480d-b932-8e1248470f71 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:03.196162 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349245f-61cc-480d-b932-8e1248470f71 None admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:03.196843 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9349245f-61cc-480d-b932-8e1248470f71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:36:03.197212 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9349245f-61cc-480d-b932-8e1248470f71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:36:03.198613 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9349245f-61cc-480d-b932-8e1248470f71 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:36:03.210514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 483/963] 10.222.0.17 () {56 vars in 1148 bytes} [Mon Oct 17 03:36:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:36:03.216838 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18263734-0789-4ad4-ac60-ce5cb3fbd5e9 None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:36:03.307689 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:36:03.313311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18263734-0789-4ad4-ac60-ce5cb3fbd5e9 None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:36:03.346708 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b3657b-2cbb-4abc-a3b6-6b0a1ed3c9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:36:03.347300 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3657b-2cbb-4abc-a3b6-6b0a1ed3c9ca None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:03.347578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3657b-2cbb-4abc-a3b6-6b0a1ed3c9ca None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:03.347889 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b3657b-2cbb-4abc-a3b6-6b0a1ed3c9ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:03.348612 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b3657b-2cbb-4abc-a3b6-6b0a1ed3c9ca None admin] RBAC: Adding query filter params (role.id=611120afb3a544d0b5161da1b769a2f8, scope.project.id=932cdb9ef7644bf5ba57c6d1d6bce7ff, user.id=b500a113a2aa4a95875d7e094b917db3) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:36:03.348919 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b3657b-2cbb-4abc-a3b6-6b0a1ed3c9ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:36:03.350327 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b3657b-2cbb-4abc-a3b6-6b0a1ed3c9ca None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:36:03.355897 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 481/964] 10.222.0.17 () {56 vars in 1374 bytes} [Mon Oct 17 03:36:03 2022] GET /identity/v3/role_assignments?user.id=b500a113a2aa4a95875d7e094b917db3&scope.project.id=932cdb9ef7644bf5ba57c6d1d6bce7ff&role.id=611120afb3a544d0b5161da1b769a2f8 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:36:05.465216 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eaf257-f285-457d-88cb-6bf1b07e528e None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:05.465798 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eaf257-f285-457d-88cb-6bf1b07e528e None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:05.465798 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eaf257-f285-457d-88cb-6bf1b07e528e None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:05.467082 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 484/965] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:36:05.472560 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08b8dd-3ebb-4c82-a62d-0da4bd69887e None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:05.472978 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08b8dd-3ebb-4c82-a62d-0da4bd69887e None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:05.473112 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08b8dd-3ebb-4c82-a62d-0da4bd69887e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:05.491341 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-cd08b8dd-3ebb-4c82-a62d-0da4bd69887e None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:36:05.492630 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-cd08b8dd-3ebb-4c82-a62d-0da4bd69887e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:36:05.557436 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 482/966] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:36:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:36:05.825763 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eaf257-f285-457d-88cb-6bf1b07e528e None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:36:05.856895 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:36:05.925784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eaf257-f285-457d-88cb-6bf1b07e528e None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:36:05.957400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495d7ad-a5c5-4dc8-aba4-13b4d353c7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:36:05.958022 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495d7ad-a5c5-4dc8-aba4-13b4d353c7d7 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:05.958303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495d7ad-a5c5-4dc8-aba4-13b4d353c7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:05.958829 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495d7ad-a5c5-4dc8-aba4-13b4d353c7d7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:05.959402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495d7ad-a5c5-4dc8-aba4-13b4d353c7d7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:36:05.960815 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495d7ad-a5c5-4dc8-aba4-13b4d353c7d7 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:36:05.969320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-7495d7ad-a5c5-4dc8-aba4-13b4d353c7d7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 03:36:05.969320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:36:05.969320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:36:05.969320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:36:05.969320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:36:05.969320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:36:05.969320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:36:05.969320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:36:05.969320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 03:36:05.969320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:36:05.969320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:36:05.969320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:36:05.969320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:36:05.969320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:36:05.969320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:36:05.969320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:36:05.969320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:36:05.971096 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:36:05.971096 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:36:05.971096 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:36:05.971096 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:36:05.971096 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:36:05.971096 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:36:05.971096 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:36:05.971096 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:36:05.971096 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:36:05.971096 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:36:05.971096 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:36:05.971096 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:36:05.971096 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:36:05.971096 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:36:05.971096 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:36:05.971096 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:36:05.971969 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:36:05.971969 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:36:05.971969 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:36:05.971969 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:36:05.971969 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:36:05.971969 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:36:05.971969 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:36:05.971969 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:36:05.971969 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:36:05.971969 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:36:05.971969 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:36:05.971969 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:36:05.971969 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 03:36:05.971969 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:36:05.971969 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 485/967] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Oct 17 03:36:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:36:05.976940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08b8dd-3ebb-4c82-a62d-0da4bd69887e None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:36:06.064200 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:36:06.073466 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08b8dd-3ebb-4c82-a62d-0da4bd69887e None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:36:06.106727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0e188-bc02-4456-b5c7-d3105819a986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:36:06.107265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0e188-bc02-4456-b5c7-d3105819a986 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:06.107554 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0e188-bc02-4456-b5c7-d3105819a986 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:06.107857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0e188-bc02-4456-b5c7-d3105819a986 None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:06.108485 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0e188-bc02-4456-b5c7-d3105819a986 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:36:06.108787 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0e188-bc02-4456-b5c7-d3105819a986 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:36:06.110778 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0e188-bc02-4456-b5c7-d3105819a986 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:36:06.115516 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 483/968] 10.222.0.17 () {56 vars in 1116 bytes} [Mon Oct 17 03:36:05 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:36:06.121352 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495d7ad-a5c5-4dc8-aba4-13b4d353c7d7 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:36:06.212742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:36:06.218215 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495d7ad-a5c5-4dc8-aba4-13b4d353c7d7 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:36:06.250347 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51366e3-6198-49ec-b94d-50653731be5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:36:06.250942 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51366e3-6198-49ec-b94d-50653731be5a None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:06.251222 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51366e3-6198-49ec-b94d-50653731be5a None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:06.251495 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51366e3-6198-49ec-b94d-50653731be5a None admin] PATH_INFO: `/v3/services` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:06.252154 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51366e3-6198-49ec-b94d-50653731be5a None admin] RBAC: Adding query filter params (type=placement) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:36:06.252454 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51366e3-6198-49ec-b94d-50653731be5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:36:06.253800 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51366e3-6198-49ec-b94d-50653731be5a None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:36:06.258004 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 486/969] 10.222.0.17 () {56 vars in 1116 bytes} [Mon Oct 17 03:36:06 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:36:08.406365 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710cfa4-e50e-47b3-ac92-c31feb9b9266 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:08.406953 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710cfa4-e50e-47b3-ac92-c31feb9b9266 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:08.406953 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710cfa4-e50e-47b3-ac92-c31feb9b9266 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:08.408306 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 484/970] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:36:08.413547 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d122ead-92bb-4a17-9872-1925b73e4084 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:08.413956 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d122ead-92bb-4a17-9872-1925b73e4084 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:08.414091 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d122ead-92bb-4a17-9872-1925b73e4084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:08.432275 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-3d122ead-92bb-4a17-9872-1925b73e4084 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:36:08.433468 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-3d122ead-92bb-4a17-9872-1925b73e4084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:36:08.499046 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 487/971] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:36:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:36:08.765365 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710cfa4-e50e-47b3-ac92-c31feb9b9266 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:36:08.864239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710cfa4-e50e-47b3-ac92-c31feb9b9266 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:36:08.896536 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9df1bfb-ebf2-4ccf-8fc2-a666b462ff04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:36:08.897142 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df1bfb-ebf2-4ccf-8fc2-a666b462ff04 None admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:08.897422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df1bfb-ebf2-4ccf-8fc2-a666b462ff04 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:08.897738 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df1bfb-ebf2-4ccf-8fc2-a666b462ff04 None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:08.898392 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df1bfb-ebf2-4ccf-8fc2-a666b462ff04 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:36:08.899892 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9df1bfb-ebf2-4ccf-8fc2-a666b462ff04 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:36:08.977050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 485/972] 10.222.0.17 () {60 vars in 1141 bytes} [Mon Oct 17 03:36:08 2022] POST /identity/v3/services => generated 246 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:36:11.103069 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07cee3-ff2f-411e-bd99-88bcc27e0fbc None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:11.103623 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07cee3-ff2f-411e-bd99-88bcc27e0fbc None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:11.103623 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07cee3-ff2f-411e-bd99-88bcc27e0fbc None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:11.104989 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 488/973] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:36:11.110759 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a16e6-1092-4eed-a790-f3d01344028f None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:11.114875 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a16e6-1092-4eed-a790-f3d01344028f None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:11.114875 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34a16e6-1092-4eed-a790-f3d01344028f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:11.130875 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-e34a16e6-1092-4eed-a790-f3d01344028f None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:36:11.132116 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-e34a16e6-1092-4eed-a790-f3d01344028f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:36:11.143076 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:36:11.215442 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 486/974] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:36:11 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:36:11.479792 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07cee3-ff2f-411e-bd99-88bcc27e0fbc None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:36:11.569431 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:36:11.579092 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07cee3-ff2f-411e-bd99-88bcc27e0fbc None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:36:11.612528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb95264-b7a6-4c9c-930a-b3c1f6bd0335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:36:11.613153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb95264-b7a6-4c9c-930a-b3c1f6bd0335 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:11.613432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb95264-b7a6-4c9c-930a-b3c1f6bd0335 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:11.613760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb95264-b7a6-4c9c-930a-b3c1f6bd0335 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:11.614336 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb95264-b7a6-4c9c-930a-b3c1f6bd0335 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:36:11.615759 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb95264-b7a6-4c9c-930a-b3c1f6bd0335 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:36:11.624443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-5eb95264-b7a6-4c9c-930a-b3c1f6bd0335 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 03:36:11.624443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:36:11.624443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:36:11.624443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:36:11.624443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:36:11.624443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:36:11.624443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:36:11.624443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:36:11.624443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 03:36:11.624443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:36:11.624443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:36:11.624443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:36:11.624443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:36:11.624443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:36:11.624443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:36:11.624443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:36:11.624443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:36:11.626098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:36:11.626098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:36:11.626098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:36:11.626098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:36:11.626098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:36:11.626098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:36:11.626098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:36:11.626098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:36:11.626098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:36:11.626098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:36:11.626098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:36:11.626098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:36:11.626098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:36:11.626098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:36:11.626098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:36:11.626098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:36:11.626953 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:36:11.626953 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:36:11.626953 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:36:11.626953 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:36:11.626953 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:36:11.626953 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:36:11.626953 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:36:11.626953 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:36:11.626953 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:36:11.626953 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:36:11.626953 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:36:11.626953 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:36:11.626953 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 03:36:11.626953 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:36:11.626953 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 489/975] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Oct 17 03:36:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:36:11.632346 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a16e6-1092-4eed-a790-f3d01344028f None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:36:11.729477 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34a16e6-1092-4eed-a790-f3d01344028f None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:36:11.763192 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b66e8-2a8d-4abb-9716-9d269702e84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:36:11.763848 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b66e8-2a8d-4abb-9716-9d269702e84c None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:11.764091 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b66e8-2a8d-4abb-9716-9d269702e84c None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:11.764369 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b66e8-2a8d-4abb-9716-9d269702e84c None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:11.764991 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21b66e8-2a8d-4abb-9716-9d269702e84c None admin] RBAC: Adding query filter params (name=placement) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:36:11.765289 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21b66e8-2a8d-4abb-9716-9d269702e84c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:36:11.766651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21b66e8-2a8d-4abb-9716-9d269702e84c None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:36:11.771623 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 487/976] 10.222.0.17 () {56 vars in 1116 bytes} [Mon Oct 17 03:36:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:36:11.777673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb95264-b7a6-4c9c-930a-b3c1f6bd0335 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:36:11.875568 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb95264-b7a6-4c9c-930a-b3c1f6bd0335 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:36:11.908254 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b08219e-008b-4db7-b7eb-24a6a56f6478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:36:11.908854 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08219e-008b-4db7-b7eb-24a6a56f6478 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:11.909133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08219e-008b-4db7-b7eb-24a6a56f6478 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:11.909406 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08219e-008b-4db7-b7eb-24a6a56f6478 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:11.910104 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b08219e-008b-4db7-b7eb-24a6a56f6478 None admin] RBAC: Adding query filter params (interface=public, service_id=5fa8c78d5ccb42cdad8665c18a92c4f1, region_id=RegionOne) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:36:11.910410 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b08219e-008b-4db7-b7eb-24a6a56f6478 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:36:11.911772 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b08219e-008b-4db7-b7eb-24a6a56f6478 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:36:11.916587 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 490/977] 10.222.0.17 () {56 vars in 1251 bytes} [Mon Oct 17 03:36:11 2022] GET /identity/v3/endpoints?service_id=5fa8c78d5ccb42cdad8665c18a92c4f1&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:36:11.922926 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b66e8-2a8d-4abb-9716-9d269702e84c None admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:36:11.926768 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:36:12.019605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21b66e8-2a8d-4abb-9716-9d269702e84c None admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:36:12.052671 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6375d-88d4-4561-a83e-93cdfb1749e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:36:12.053261 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6375d-88d4-4561-a83e-93cdfb1749e8 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:12.053539 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6375d-88d4-4561-a83e-93cdfb1749e8 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:12.053808 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6375d-88d4-4561-a83e-93cdfb1749e8 None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:12.054396 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c6375d-88d4-4561-a83e-93cdfb1749e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:36:12.055755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c6375d-88d4-4561-a83e-93cdfb1749e8 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:36:12.062340 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 488/978] 10.222.0.17 () {56 vars in 1087 bytes} [Mon Oct 17 03:36:11 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:36:14.199060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c364bd8-84a9-4f96-b1cd-cb1bb57d4c28 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:14.199642 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c364bd8-84a9-4f96-b1cd-cb1bb57d4c28 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:14.199642 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c364bd8-84a9-4f96-b1cd-cb1bb57d4c28 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:14.200965 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 491/979] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:36:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:36:14.206894 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654bb54c-7ac2-4435-8cad-35335f7c5e58 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:14.207285 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654bb54c-7ac2-4435-8cad-35335f7c5e58 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:14.207448 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654bb54c-7ac2-4435-8cad-35335f7c5e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:14.225647 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-654bb54c-7ac2-4435-8cad-35335f7c5e58 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:36:14.226825 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-654bb54c-7ac2-4435-8cad-35335f7c5e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:36:14.287666 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:36:14.292961 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 489/980] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:36:14 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 17 03:36:14.558371 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c364bd8-84a9-4f96-b1cd-cb1bb57d4c28 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:36:14.577852 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:36:14.657731 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c364bd8-84a9-4f96-b1cd-cb1bb57d4c28 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:36:14.691815 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dacb8a-e4db-48e3-900c-1fb909cfb893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:36:14.692442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dacb8a-e4db-48e3-900c-1fb909cfb893 None admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:14.692715 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dacb8a-e4db-48e3-900c-1fb909cfb893 None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:14.692977 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dacb8a-e4db-48e3-900c-1fb909cfb893 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:14.693548 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dacb8a-e4db-48e3-900c-1fb909cfb893 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:36:14.694923 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76dacb8a-e4db-48e3-900c-1fb909cfb893 None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:36:14.703514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application [None req-76dacb8a-e4db-48e3-900c-1fb909cfb893 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 03:36:14.703514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:36:14.703514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 17 03:36:14.703514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 17 03:36:14.703514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 17 03:36:14.703514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 17 03:36:14.703514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 17 03:36:14.703514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:36:14.703514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 03:36:14.703514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:36:14.703514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 17 03:36:14.703514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:36:14.703514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 17 03:36:14.703514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 17 03:36:14.703514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 17 03:36:14.703514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 17 03:36:14.703514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 17 03:36:14.704869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 17 03:36:14.704869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 17 03:36:14.704869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 17 03:36:14.704869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 17 03:36:14.704869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 17 03:36:14.704869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 17 03:36:14.704869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 17 03:36:14.704869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 17 03:36:14.704869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 17 03:36:14.704869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 17 03:36:14.704869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 17 03:36:14.704869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 17 03:36:14.704869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 17 03:36:14.704869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 17 03:36:14.704869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 17 03:36:14.704869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.get_or_create( Oct 17 03:36:14.706331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 17 03:36:14.706331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application with Lock( Oct 17 03:36:14.706331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 17 03:36:14.706331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self._enter() Oct 17 03:36:14.706331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 17 03:36:14.706331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 17 03:36:14.706331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 17 03:36:14.706331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application return self.creator() Oct 17 03:36:14.706331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 17 03:36:14.706331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application created_value = creator( Oct 17 03:36:14.706331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 17 03:36:14.706331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 17 03:36:14.706331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 17 03:36:14.706331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: ERROR keystone.server.flask.application Oct 17 03:36:14.706331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 492/981] 10.222.0.17 () {56 vars in 1117 bytes} [Mon Oct 17 03:36:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 17 03:36:14.711878 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654bb54c-7ac2-4435-8cad-35335f7c5e58 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:36:14.808355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654bb54c-7ac2-4435-8cad-35335f7c5e58 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:36:14.841893 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a9876-899f-4531-92f4-889d85adc679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:36:14.842433 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a9876-899f-4531-92f4-889d85adc679 None admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:14.842709 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a9876-899f-4531-92f4-889d85adc679 None admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:14.842977 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a9876-899f-4531-92f4-889d85adc679 None admin] PATH_INFO: `/v3/services` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:14.843595 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a9876-899f-4531-92f4-889d85adc679 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:36:14.844637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a9876-899f-4531-92f4-889d85adc679 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:36:14.846701 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a9876-899f-4531-92f4-889d85adc679 None admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:36:14.851913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 490/982] 10.222.0.17 () {56 vars in 1116 bytes} [Mon Oct 17 03:36:14 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:36:14.857665 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dacb8a-e4db-48e3-900c-1fb909cfb893 None admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:36:14.938666 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:36:14.955798 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76dacb8a-e4db-48e3-900c-1fb909cfb893 None admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:36:14.989620 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad8358-69d1-4047-b10f-f84beeb1714f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:36:14.990217 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad8358-69d1-4047-b10f-f84beeb1714f None admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:36:14.990496 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad8358-69d1-4047-b10f-f84beeb1714f None admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:36:14.990766 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad8358-69d1-4047-b10f-f84beeb1714f None admin] PATH_INFO: `/v3/endpoints` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:36:14.991432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dad8358-69d1-4047-b10f-f84beeb1714f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:36:14.992808 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dad8358-69d1-4047-b10f-f84beeb1714f None admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:36:15.078766 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 493/983] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:36:14 2022] POST /identity/v3/endpoints => generated 331 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:36:24.848889 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=69 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:36:25.074459 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:42:30.010872 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc23856-8455-4168-a130-9b7caca2aa54 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:30.011372 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc23856-8455-4168-a130-9b7caca2aa54 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:30.012544 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc23856-8455-4168-a130-9b7caca2aa54 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:30.016971 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 491/984] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:42:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:42:30.020317 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d12e86-a041-4a24-9f8d-939437ef1036 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:30.021216 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d12e86-a041-4a24-9f8d-939437ef1036 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:30.021740 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d12e86-a041-4a24-9f8d-939437ef1036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:30.120178 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-f5d12e86-a041-4a24-9f8d-939437ef1036 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:42:30.122706 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-f5d12e86-a041-4a24-9f8d-939437ef1036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:30.240890 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 494/985] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:42:30.532565 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ff902-4cf3-4a56-8d61-5f95506aba07 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:30.532917 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ff902-4cf3-4a56-8d61-5f95506aba07 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:30.533263 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5ff902-4cf3-4a56-8d61-5f95506aba07 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:30.534733 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 492/986] 10.222.0.17 () {54 vars in 964 bytes} [Mon Oct 17 03:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:42:30.541077 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8de033-826d-4b2a-b799-90161c2dfa5e None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:30.541431 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8de033-826d-4b2a-b799-90161c2dfa5e None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:30.541774 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8de033-826d-4b2a-b799-90161c2dfa5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:30.621798 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-6c8de033-826d-4b2a-b799-90161c2dfa5e None None] MFA Rules not processed for user `414707c545f641d3935720356b326672`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:42:30.623136 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-6c8de033-826d-4b2a-b799-90161c2dfa5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:30.644055 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:42:30.708979 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 495/987] 10.222.0.17 () {58 vars in 1061 bytes} [Mon Oct 17 03:42:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:42:30.718508 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ff902-4cf3-4a56-8d61-5f95506aba07 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:42:30.720375 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-8f5ff902-4cf3-4a56-8d61-5f95506aba07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:30.830088 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5ff902-4cf3-4a56-8d61-5f95506aba07 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:42:30.927650 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd245d-3da9-4779-b422-073012c4885e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:42:30.928502 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd245d-3da9-4779-b422-073012c4885e service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:30.929007 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd245d-3da9-4779-b422-073012c4885e service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:30.930073 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd245d-3da9-4779-b422-073012c4885e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:30.931994 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6bfd245d-3da9-4779-b422-073012c4885e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:30.999967 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:42:31.064757 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfd245d-3da9-4779-b422-073012c4885e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:42:31.067069 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfd245d-3da9-4779-b422-073012c4885e service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:42:31.091523 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 493/988] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:42:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:42:33.583865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416dc67d-d004-4c96-890e-e7d01f9a1854 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:33.584427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416dc67d-d004-4c96-890e-e7d01f9a1854 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:33.584427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416dc67d-d004-4c96-890e-e7d01f9a1854 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:33.585764 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 496/989] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:42:33.591141 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38978b-79c0-4eb5-862e-ffc934a4b92c None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:33.591558 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38978b-79c0-4eb5-862e-ffc934a4b92c None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:33.591695 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38978b-79c0-4eb5-862e-ffc934a4b92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:33.661131 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-1f38978b-79c0-4eb5-862e-ffc934a4b92c None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:42:33.662343 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-1f38978b-79c0-4eb5-862e-ffc934a4b92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:33.758073 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 494/990] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:42:34.044099 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416dc67d-d004-4c96-890e-e7d01f9a1854 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:42:34.045626 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-416dc67d-d004-4c96-890e-e7d01f9a1854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:34.148267 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416dc67d-d004-4c96-890e-e7d01f9a1854 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:42:34.217678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:42:34.219948 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b05307-53b4-401c-95e8-dc49bdb4287a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:42:34.220548 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b05307-53b4-401c-95e8-dc49bdb4287a service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:34.220821 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b05307-53b4-401c-95e8-dc49bdb4287a service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:34.221086 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b05307-53b4-401c-95e8-dc49bdb4287a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:34.222656 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-61b05307-53b4-401c-95e8-dc49bdb4287a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:34.329463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b05307-53b4-401c-95e8-dc49bdb4287a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:42:34.330818 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b05307-53b4-401c-95e8-dc49bdb4287a service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:42:34.349257 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 497/991] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:42:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:42:37.009641 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a64d8-3c15-4654-9bd7-387354d5db5c None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:37.010213 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a64d8-3c15-4654-9bd7-387354d5db5c None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:37.010213 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a64d8-3c15-4654-9bd7-387354d5db5c None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:37.011633 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 495/992] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:42:37.016927 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fb052-3964-4d37-95f5-35205c9ea8ef None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:37.017362 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fb052-3964-4d37-95f5-35205c9ea8ef None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:37.017467 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541fb052-3964-4d37-95f5-35205c9ea8ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:37.081945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-541fb052-3964-4d37-95f5-35205c9ea8ef None None] MFA Rules not processed for user `6704615fe74b48a4a33d01b3505139b8`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:42:37.083138 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-541fb052-3964-4d37-95f5-35205c9ea8ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:37.134754 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:42:37.179086 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 498/993] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:42:37 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:42:37.467575 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a64d8-3c15-4654-9bd7-387354d5db5c None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:42:37.469134 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-3f2a64d8-3c15-4654-9bd7-387354d5db5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:37.479772 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:42:37.571239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a64d8-3c15-4654-9bd7-387354d5db5c None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:42:37.644163 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbceca8-2b30-4d38-8f55-40fa20f2ed13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:42:37.644764 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbceca8-2b30-4d38-8f55-40fa20f2ed13 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:37.645044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbceca8-2b30-4d38-8f55-40fa20f2ed13 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:37.645317 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbceca8-2b30-4d38-8f55-40fa20f2ed13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:37.646914 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-cfbceca8-2b30-4d38-8f55-40fa20f2ed13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:37.708573 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:42:37.753841 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbceca8-2b30-4d38-8f55-40fa20f2ed13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:42:37.755314 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbceca8-2b30-4d38-8f55-40fa20f2ed13 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:42:37.774100 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 496/994] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:42:37 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:42:40.666225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c2304-8410-42f6-92e7-6e8727b4a9b3 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:40.666784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c2304-8410-42f6-92e7-6e8727b4a9b3 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:40.666784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c2304-8410-42f6-92e7-6e8727b4a9b3 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:40.668723 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 499/995] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:42:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:42:40.674128 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b5abe-523b-4608-a9e0-84906804e55b None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:40.674549 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b5abe-523b-4608-a9e0-84906804e55b None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:40.674658 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b5abe-523b-4608-a9e0-84906804e55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:40.739769 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-830b5abe-523b-4608-a9e0-84906804e55b None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:42:40.740969 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-830b5abe-523b-4608-a9e0-84906804e55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:40.834568 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 497/996] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:42:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:42:41.131626 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb604e5-266b-4100-baa0-16af86614a59 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:41.131942 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb604e5-266b-4100-baa0-16af86614a59 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:41.132218 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb604e5-266b-4100-baa0-16af86614a59 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:41.133612 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 500/997] 10.222.0.17 () {54 vars in 964 bytes} [Mon Oct 17 03:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:42:41.139448 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda661da-66c6-4ddf-9b47-5461cf7a1479 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:41.139749 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda661da-66c6-4ddf-9b47-5461cf7a1479 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:41.142074 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda661da-66c6-4ddf-9b47-5461cf7a1479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:41.162298 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:42:41.215823 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-cda661da-66c6-4ddf-9b47-5461cf7a1479 None None] MFA Rules not processed for user `414707c545f641d3935720356b326672`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:42:41.217031 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-cda661da-66c6-4ddf-9b47-5461cf7a1479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:41.291289 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 498/998] 10.222.0.17 () {58 vars in 1061 bytes} [Mon Oct 17 03:42:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:42:41.301025 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb604e5-266b-4100-baa0-16af86614a59 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:42:41.302520 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-acb604e5-266b-4100-baa0-16af86614a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:41.401865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb604e5-266b-4100-baa0-16af86614a59 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:42:41.472932 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5443ba2-7f58-4325-bb06-8226faae28d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:42:41.473523 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5443ba2-7f58-4325-bb06-8226faae28d0 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:41.473800 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5443ba2-7f58-4325-bb06-8226faae28d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:41.474068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5443ba2-7f58-4325-bb06-8226faae28d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:41.475618 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-e5443ba2-7f58-4325-bb06-8226faae28d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:41.482247 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:42:41.580577 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5443ba2-7f58-4325-bb06-8226faae28d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:42:41.581984 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5443ba2-7f58-4325-bb06-8226faae28d0 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:42:41.600018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 501/999] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:42:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:42:45.214523 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdde44a-ded8-4e76-9b53-bd042eb03412 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:45.215025 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdde44a-ded8-4e76-9b53-bd042eb03412 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:45.215125 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdde44a-ded8-4e76-9b53-bd042eb03412 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:45.216800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 499/1000] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:42:45.222098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531161c-178a-4fdb-a413-253363f49d31 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:45.222512 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531161c-178a-4fdb-a413-253363f49d31 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:45.222651 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8531161c-178a-4fdb-a413-253363f49d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:45.296727 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-8531161c-178a-4fdb-a413-253363f49d31 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:42:45.297951 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-8531161c-178a-4fdb-a413-253363f49d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:45.360093 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:42:45.402584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 502/1001] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:42:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:42:45.684981 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdde44a-ded8-4e76-9b53-bd042eb03412 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:42:45.686599 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-4bdde44a-ded8-4e76-9b53-bd042eb03412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:45.798892 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdde44a-ded8-4e76-9b53-bd042eb03412 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:42:45.813025 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:42:45.873536 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c12e49-0807-4715-8ac0-749cda45587b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:42:45.874150 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c12e49-0807-4715-8ac0-749cda45587b service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:45.874429 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c12e49-0807-4715-8ac0-749cda45587b service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:45.874715 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c12e49-0807-4715-8ac0-749cda45587b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:45.877208 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-21c12e49-0807-4715-8ac0-749cda45587b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:45.989263 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c12e49-0807-4715-8ac0-749cda45587b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:42:45.990721 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c12e49-0807-4715-8ac0-749cda45587b service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:42:46.009495 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 500/1002] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:42:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:42:48.215184 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f8d81-f0c5-4bfe-8a33-b0da24c8f815 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:48.215991 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f8d81-f0c5-4bfe-8a33-b0da24c8f815 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:48.216084 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f8d81-f0c5-4bfe-8a33-b0da24c8f815 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:48.217358 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 503/1003] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:42:48.222654 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d20c3-2fee-4812-a641-2d98723b145c None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:48.223067 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d20c3-2fee-4812-a641-2d98723b145c None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:48.223318 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d20c3-2fee-4812-a641-2d98723b145c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:48.270844 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:42:48.289747 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-bd8d20c3-2fee-4812-a641-2d98723b145c None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:42:48.295929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-bd8d20c3-2fee-4812-a641-2d98723b145c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:48.395278 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 501/1004] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:42:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:42:48.673265 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f8d81-f0c5-4bfe-8a33-b0da24c8f815 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:42:48.674800 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-b81f8d81-f0c5-4bfe-8a33-b0da24c8f815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:48.777467 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f8d81-f0c5-4bfe-8a33-b0da24c8f815 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:42:48.849901 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c9c3da-aac6-45c5-ac34-53f182e21fb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:42:48.850522 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9c3da-aac6-45c5-ac34-53f182e21fb5 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:48.850799 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9c3da-aac6-45c5-ac34-53f182e21fb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:48.851068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9c3da-aac6-45c5-ac34-53f182e21fb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:48.852696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-a1c9c3da-aac6-45c5-ac34-53f182e21fb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:48.869960 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:42:48.960959 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c9c3da-aac6-45c5-ac34-53f182e21fb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:42:48.962315 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c9c3da-aac6-45c5-ac34-53f182e21fb5 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:42:48.981213 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 504/1005] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:42:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:42:51.179320 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf25e2-882a-47d6-a4e5-e7954d942ee0 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:51.179841 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf25e2-882a-47d6-a4e5-e7954d942ee0 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:51.179841 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf25e2-882a-47d6-a4e5-e7954d942ee0 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:51.181311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 502/1006] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:42:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:42:51.187300 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3073b8-ff22-460f-bc96-bd4c9e45477b None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:51.187712 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3073b8-ff22-460f-bc96-bd4c9e45477b None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:51.187915 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3073b8-ff22-460f-bc96-bd4c9e45477b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:51.263218 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-4d3073b8-ff22-460f-bc96-bd4c9e45477b None None] MFA Rules not processed for user `6704615fe74b48a4a33d01b3505139b8`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:42:51.264522 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-4d3073b8-ff22-460f-bc96-bd4c9e45477b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:51.331279 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:42:51.363391 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 505/1007] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:42:51 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:42:51.649160 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf25e2-882a-47d6-a4e5-e7954d942ee0 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:42:51.650794 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-25cf25e2-882a-47d6-a4e5-e7954d942ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:51.760385 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf25e2-882a-47d6-a4e5-e7954d942ee0 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:42:51.781038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:42:51.836154 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee846d-b321-444e-a5b1-925e964efadf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:42:51.836756 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee846d-b321-444e-a5b1-925e964efadf service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:51.837034 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee846d-b321-444e-a5b1-925e964efadf service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:51.837304 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee846d-b321-444e-a5b1-925e964efadf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:51.839532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-1eee846d-b321-444e-a5b1-925e964efadf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:51.952281 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee846d-b321-444e-a5b1-925e964efadf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:42:51.953714 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee846d-b321-444e-a5b1-925e964efadf service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:42:51.972831 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 503/1008] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:42:51 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:42:54.512708 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93cb52-a564-4115-93dd-80d8962c7833 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:54.513269 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93cb52-a564-4115-93dd-80d8962c7833 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:54.513269 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb93cb52-a564-4115-93dd-80d8962c7833 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:54.514627 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 506/1009] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:42:54.519909 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250cb98-5dbf-4e6e-ac56-ff4d2cc68d04 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:54.520299 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250cb98-5dbf-4e6e-ac56-ff4d2cc68d04 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:54.520454 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250cb98-5dbf-4e6e-ac56-ff4d2cc68d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:54.582162 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:42:54.584980 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-5250cb98-5dbf-4e6e-ac56-ff4d2cc68d04 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:42:54.586207 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-5250cb98-5dbf-4e6e-ac56-ff4d2cc68d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:54.682247 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 504/1010] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:42:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:42:54.964708 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93cb52-a564-4115-93dd-80d8962c7833 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:42:54.966567 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-eb93cb52-a564-4115-93dd-80d8962c7833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:55.068987 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb93cb52-a564-4115-93dd-80d8962c7833 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:42:55.140788 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ca2bd-9731-4867-bb03-8307df1addf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:42:55.141385 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ca2bd-9731-4867-bb03-8307df1addf9 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:55.141659 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ca2bd-9731-4867-bb03-8307df1addf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:55.141926 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ca2bd-9731-4867-bb03-8307df1addf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:55.143494 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-676ca2bd-9731-4867-bb03-8307df1addf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:55.169407 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:42:55.249516 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ca2bd-9731-4867-bb03-8307df1addf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:42:55.250925 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ca2bd-9731-4867-bb03-8307df1addf9 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:42:55.268665 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 507/1011] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:42:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:42:57.446138 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915fd46-d401-4850-a5d7-0afede05a30d None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:57.446648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915fd46-d401-4850-a5d7-0afede05a30d None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:57.446648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7915fd46-d401-4850-a5d7-0afede05a30d None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:57.448478 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 505/1012] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:42:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:42:57.453853 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690cd9e1-3f77-455f-82aa-a993043522e7 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:57.454265 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690cd9e1-3f77-455f-82aa-a993043522e7 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:57.454406 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690cd9e1-3f77-455f-82aa-a993043522e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:57.519141 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-690cd9e1-3f77-455f-82aa-a993043522e7 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:42:57.520345 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-690cd9e1-3f77-455f-82aa-a993043522e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:57.592796 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:42:57.613824 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 508/1013] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:42:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:42:57.899917 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915fd46-d401-4850-a5d7-0afede05a30d None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:42:57.901563 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-7915fd46-d401-4850-a5d7-0afede05a30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:58.004613 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7915fd46-d401-4850-a5d7-0afede05a30d None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:42:58.032728 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:42:58.078146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3cd4e8-f3d9-4bf3-8777-f9b676621dea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:42:58.078741 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3cd4e8-f3d9-4bf3-8777-f9b676621dea service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:42:58.079019 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3cd4e8-f3d9-4bf3-8777-f9b676621dea service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:42:58.079290 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3cd4e8-f3d9-4bf3-8777-f9b676621dea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:42:58.080907 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-dc3cd4e8-f3d9-4bf3-8777-f9b676621dea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:42:58.186073 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3cd4e8-f3d9-4bf3-8777-f9b676621dea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:42:58.187438 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3cd4e8-f3d9-4bf3-8777-f9b676621dea service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:42:58.205915 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 506/1014] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:42:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:43:00.859077 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a38fb-4cc3-4850-bd10-bbd867b429c3 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:00.859607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a38fb-4cc3-4850-bd10-bbd867b429c3 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:00.859702 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a38fb-4cc3-4850-bd10-bbd867b429c3 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:00.861020 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 509/1015] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:43:00.866057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037deb75-841a-4527-a679-95c07efe0a4c None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:00.866475 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037deb75-841a-4527-a679-95c07efe0a4c None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:00.866608 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037deb75-841a-4527-a679-95c07efe0a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:00.932615 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-037deb75-841a-4527-a679-95c07efe0a4c None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:43:00.933838 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-037deb75-841a-4527-a679-95c07efe0a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:00.942094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:01.028174 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 507/1016] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:43:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:43:01.309495 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a38fb-4cc3-4850-bd10-bbd867b429c3 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:43:01.311034 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-1b3a38fb-4cc3-4850-bd10-bbd867b429c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:01.417674 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a38fb-4cc3-4850-bd10-bbd867b429c3 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:43:01.490179 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06828ec-e402-4243-83a9-b90c1cff5198 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:43:01.490786 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06828ec-e402-4243-83a9-b90c1cff5198 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:01.491065 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06828ec-e402-4243-83a9-b90c1cff5198 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:01.491335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06828ec-e402-4243-83a9-b90c1cff5198 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:01.492949 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-e06828ec-e402-4243-83a9-b90c1cff5198 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:01.528576 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:01.600990 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06828ec-e402-4243-83a9-b90c1cff5198 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:43:01.602397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06828ec-e402-4243-83a9-b90c1cff5198 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:43:01.621356 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 510/1017] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:43:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:43:05.941296 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa88f74-cb60-4cd3-b6d8-b3cc5c54c89d None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:05.941878 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa88f74-cb60-4cd3-b6d8-b3cc5c54c89d None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:05.941878 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa88f74-cb60-4cd3-b6d8-b3cc5c54c89d None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:05.943389 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 508/1018] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:43:05.948448 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5788b5d-8eff-49b8-8343-08cc48980bbd None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:05.948851 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5788b5d-8eff-49b8-8343-08cc48980bbd None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:05.949005 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5788b5d-8eff-49b8-8343-08cc48980bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:06.015354 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-c5788b5d-8eff-49b8-8343-08cc48980bbd None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:43:06.016574 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c5788b5d-8eff-49b8-8343-08cc48980bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:06.104669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:06.116959 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 511/1019] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:43:06.405995 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa88f74-cb60-4cd3-b6d8-b3cc5c54c89d None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:43:06.407669 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-8aa88f74-cb60-4cd3-b6d8-b3cc5c54c89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:06.511911 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa88f74-cb60-4cd3-b6d8-b3cc5c54c89d None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:43:06.550867 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:06.588359 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95777813-ba87-488a-88ae-a4362b32b44c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:43:06.588969 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95777813-ba87-488a-88ae-a4362b32b44c service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:06.589245 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95777813-ba87-488a-88ae-a4362b32b44c service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:06.589509 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95777813-ba87-488a-88ae-a4362b32b44c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:06.591339 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-95777813-ba87-488a-88ae-a4362b32b44c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:06.700436 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95777813-ba87-488a-88ae-a4362b32b44c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:43:06.701876 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95777813-ba87-488a-88ae-a4362b32b44c service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:43:06.720667 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 509/1020] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:43:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:43:09.881426 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983cb01-f972-469f-b56b-1519e321d914 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:09.881986 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983cb01-f972-469f-b56b-1519e321d914 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:09.881986 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e983cb01-f972-469f-b56b-1519e321d914 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:09.883395 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 512/1021] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:43:09.889423 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68689ca3-8f1d-48e0-b35f-9eda1dafcf46 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:09.889938 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68689ca3-8f1d-48e0-b35f-9eda1dafcf46 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:09.890038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68689ca3-8f1d-48e0-b35f-9eda1dafcf46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:09.961474 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-68689ca3-8f1d-48e0-b35f-9eda1dafcf46 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:43:09.962590 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-68689ca3-8f1d-48e0-b35f-9eda1dafcf46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:09.981495 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:10.059188 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 510/1022] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:43:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:43:10.345454 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e983cb01-f972-469f-b56b-1519e321d914 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:43:10.347101 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-e983cb01-f972-469f-b56b-1519e321d914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:10.452423 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e983cb01-f972-469f-b56b-1519e321d914 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:43:10.527928 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b29f5-02ca-4c5f-ab20-4a56a2cc64d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:43:10.528536 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b29f5-02ca-4c5f-ab20-4a56a2cc64d9 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:10.528852 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b29f5-02ca-4c5f-ab20-4a56a2cc64d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:10.529140 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b29f5-02ca-4c5f-ab20-4a56a2cc64d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:10.530767 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-9b4b29f5-02ca-4c5f-ab20-4a56a2cc64d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:10.574675 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:10.657883 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b29f5-02ca-4c5f-ab20-4a56a2cc64d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:43:10.659420 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b29f5-02ca-4c5f-ab20-4a56a2cc64d9 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:43:10.687042 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 513/1023] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:43:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:43:15.024574 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a43ec-2758-41c3-b9d3-ecce48266d60 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:15.025082 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a43ec-2758-41c3-b9d3-ecce48266d60 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:15.025247 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a43ec-2758-41c3-b9d3-ecce48266d60 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:15.026598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 511/1024] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:43:15.032169 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ae38b-8e1d-41c4-92b5-a87602fbeb33 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:15.032590 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ae38b-8e1d-41c4-92b5-a87602fbeb33 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:15.032725 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8ae38b-8e1d-41c4-92b5-a87602fbeb33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:15.103608 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-de8ae38b-8e1d-41c4-92b5-a87602fbeb33 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:43:15.104850 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-de8ae38b-8e1d-41c4-92b5-a87602fbeb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:15.201648 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:15.204303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 514/1025] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:43:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:43:15.482535 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a43ec-2758-41c3-b9d3-ecce48266d60 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:43:15.485201 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-f03a43ec-2758-41c3-b9d3-ecce48266d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:15.592355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a43ec-2758-41c3-b9d3-ecce48266d60 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:43:15.639177 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:15.666200 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fad779d-861d-4c04-8dc3-faf3f82be2fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:43:15.666797 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad779d-861d-4c04-8dc3-faf3f82be2fe service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:15.667083 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad779d-861d-4c04-8dc3-faf3f82be2fe service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:15.667355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad779d-861d-4c04-8dc3-faf3f82be2fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:15.668993 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-5fad779d-861d-4c04-8dc3-faf3f82be2fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:15.775788 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fad779d-861d-4c04-8dc3-faf3f82be2fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:43:15.777218 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fad779d-861d-4c04-8dc3-faf3f82be2fe service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:43:15.795961 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 512/1026] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:43:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:43:20.667342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f189b7-032a-45af-a209-cc58d671dc5e None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:20.667794 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f189b7-032a-45af-a209-cc58d671dc5e None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:20.669589 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f189b7-032a-45af-a209-cc58d671dc5e None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:20.671203 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 515/1027] 10.222.0.17 () {54 vars in 921 bytes} [Mon Oct 17 03:43:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:43:20.680736 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfc1bc-b241-4927-a9a7-a330fad7e3a1 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:20.682642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfc1bc-b241-4927-a9a7-a330fad7e3a1 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:20.682924 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfc1bc-b241-4927-a9a7-a330fad7e3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:20.789701 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-8abfc1bc-b241-4927-a9a7-a330fad7e3a1 None None] MFA Rules not processed for user `b500a113a2aa4a95875d7e094b917db3`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:43:20.791191 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-8abfc1bc-b241-4927-a9a7-a330fad7e3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:20.836506 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:20.954210 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 513/1028] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Oct 17 03:43:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:43:25.125847 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9228f6-bf63-4251-9b40-7d211e56eea8 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:25.126364 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9228f6-bf63-4251-9b40-7d211e56eea8 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:25.126547 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9228f6-bf63-4251-9b40-7d211e56eea8 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:25.128041 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 516/1029] 10.222.0.17 () {54 vars in 921 bytes} [Mon Oct 17 03:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:43:25.134184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b5087-cccd-45e2-b8b0-e015bf2b9fda None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:25.134619 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b5087-cccd-45e2-b8b0-e015bf2b9fda None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:25.134772 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b5087-cccd-45e2-b8b0-e015bf2b9fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:25.216011 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-787b5087-cccd-45e2-b8b0-e015bf2b9fda None None] MFA Rules not processed for user `b500a113a2aa4a95875d7e094b917db3`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:43:25.217314 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-787b5087-cccd-45e2-b8b0-e015bf2b9fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:25.296173 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:25.332699 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 514/1030] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Oct 17 03:43:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:43:26.487923 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9229da7-d419-4a9f-99f2-69afac1f9975 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:26.488498 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9229da7-d419-4a9f-99f2-69afac1f9975 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:26.488590 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9229da7-d419-4a9f-99f2-69afac1f9975 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:26.489946 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 517/1031] 10.222.0.17 () {54 vars in 921 bytes} [Mon Oct 17 03:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:43:26.494907 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68bcc6-2b84-43e7-8657-35b2ca8cdca7 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:26.495340 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68bcc6-2b84-43e7-8657-35b2ca8cdca7 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:26.495449 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68bcc6-2b84-43e7-8657-35b2ca8cdca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:26.582295 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-cb68bcc6-2b84-43e7-8657-35b2ca8cdca7 None None] MFA Rules not processed for user `b500a113a2aa4a95875d7e094b917db3`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:43:26.584335 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-cb68bcc6-2b84-43e7-8657-35b2ca8cdca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:26.706209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 515/1032] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Oct 17 03:43:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:43:30.378434 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eda8df-72a3-46f6-bad6-defea1310f23 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:30.378939 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eda8df-72a3-46f6-bad6-defea1310f23 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:30.379159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eda8df-72a3-46f6-bad6-defea1310f23 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:30.380639 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 518/1033] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:43:30.386313 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b578b-25a7-4414-b19d-ca8e001648a8 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:30.386782 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b578b-25a7-4414-b19d-ca8e001648a8 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:30.387057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b578b-25a7-4414-b19d-ca8e001648a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:30.425037 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:30.457579 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-cd2b578b-25a7-4414-b19d-ca8e001648a8 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:43:30.458938 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-cd2b578b-25a7-4414-b19d-ca8e001648a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:30.561088 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 516/1034] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:43:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:43:30.857093 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8c0f5-63e5-44a3-affc-9922dd7cce8f None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:30.857443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8c0f5-63e5-44a3-affc-9922dd7cce8f None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:30.857651 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8c0f5-63e5-44a3-affc-9922dd7cce8f None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:30.859144 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 519/1035] 10.222.0.17 () {54 vars in 961 bytes} [Mon Oct 17 03:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:43:30.867677 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17386a-7869-47d3-9309-5c5c4eefdc45 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:30.868278 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17386a-7869-47d3-9309-5c5c4eefdc45 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:30.868658 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17386a-7869-47d3-9309-5c5c4eefdc45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:30.959619 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-dd17386a-7869-47d3-9309-5c5c4eefdc45 None None] MFA Rules not processed for user `d961b751763a4e49b5d4ca4c795c5bf6`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:43:30.961102 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-dd17386a-7869-47d3-9309-5c5c4eefdc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:31.021930 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:31.092485 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 517/1036] 10.222.0.17 () {58 vars in 1058 bytes} [Mon Oct 17 03:43:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:43:31.100353 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e8c0f5-63e5-44a3-affc-9922dd7cce8f None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:43:31.102064 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-50e8c0f5-63e5-44a3-affc-9922dd7cce8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:31.264190 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e8c0f5-63e5-44a3-affc-9922dd7cce8f None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:43:31.344765 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:31.388417 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700fe93-c7b4-4b62-97c0-685a72ca5f2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:43:31.389089 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700fe93-c7b4-4b62-97c0-685a72ca5f2b service nova] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:31.389313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700fe93-c7b4-4b62-97c0-685a72ca5f2b service nova] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:31.389559 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700fe93-c7b4-4b62-97c0-685a72ca5f2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:31.391259 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-9700fe93-c7b4-4b62-97c0-685a72ca5f2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:31.490945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9700fe93-c7b4-4b62-97c0-685a72ca5f2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:43:31.492615 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9700fe93-c7b4-4b62-97c0-685a72ca5f2b service nova] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:43:31.532121 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 520/1037] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:43:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:43:33.873654 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c0dca-34c6-4bc3-8fd4-4e0d00797914 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:33.874092 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c0dca-34c6-4bc3-8fd4-4e0d00797914 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:33.874410 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c0dca-34c6-4bc3-8fd4-4e0d00797914 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:33.875906 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 518/1038] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:43:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:43:33.885409 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1fef8-1b34-43c4-a342-727108e102d0 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:33.885866 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1fef8-1b34-43c4-a342-727108e102d0 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:33.885866 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1fef8-1b34-43c4-a342-727108e102d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:33.958933 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-76b1fef8-1b34-43c4-a342-727108e102d0 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:43:33.960189 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-76b1fef8-1b34-43c4-a342-727108e102d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:34.000966 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:34.064574 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 521/1039] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:43:34.356104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c0dca-34c6-4bc3-8fd4-4e0d00797914 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:43:34.357653 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-3c7c0dca-34c6-4bc3-8fd4-4e0d00797914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:34.534308 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c0dca-34c6-4bc3-8fd4-4e0d00797914 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:43:34.545952 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:34.682431 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d98446-4a69-4ed5-8b2b-5411bde2f5b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:43:34.683189 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d98446-4a69-4ed5-8b2b-5411bde2f5b6 service nova] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:34.683370 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d98446-4a69-4ed5-8b2b-5411bde2f5b6 service nova] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:34.683590 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d98446-4a69-4ed5-8b2b-5411bde2f5b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:34.685242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-64d98446-4a69-4ed5-8b2b-5411bde2f5b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:34.780041 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d98446-4a69-4ed5-8b2b-5411bde2f5b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:43:34.781669 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d98446-4a69-4ed5-8b2b-5411bde2f5b6 service nova] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:43:34.801149 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 519/1040] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:43:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:43:37.365978 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e809cc6-5746-4cdb-947d-bbc414151eb7 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:37.366578 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e809cc6-5746-4cdb-947d-bbc414151eb7 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:37.366782 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e809cc6-5746-4cdb-947d-bbc414151eb7 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:37.368512 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 522/1041] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:43:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:43:37.373856 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876c21e-c5b7-4579-9e76-a7468fa15d9a None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:37.374625 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876c21e-c5b7-4579-9e76-a7468fa15d9a None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:37.375140 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876c21e-c5b7-4579-9e76-a7468fa15d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:37.391078 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:37.446217 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-2876c21e-c5b7-4579-9e76-a7468fa15d9a None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:43:37.447520 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-2876c21e-c5b7-4579-9e76-a7468fa15d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:37.550088 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 520/1042] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:43:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:43:37.838610 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405f078-5b76-4789-82b9-92b4598aba16 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:37.839079 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405f078-5b76-4789-82b9-92b4598aba16 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:37.839474 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405f078-5b76-4789-82b9-92b4598aba16 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:37.841329 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 523/1043] 10.222.0.17 () {54 vars in 961 bytes} [Mon Oct 17 03:43:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:43:37.846490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726c8c8-1599-479e-bf64-c59f96ee4935 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:37.846777 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726c8c8-1599-479e-bf64-c59f96ee4935 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:37.847052 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2726c8c8-1599-479e-bf64-c59f96ee4935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:37.924742 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-2726c8c8-1599-479e-bf64-c59f96ee4935 None None] MFA Rules not processed for user `d961b751763a4e49b5d4ca4c795c5bf6`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:43:37.925926 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-2726c8c8-1599-479e-bf64-c59f96ee4935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:37.962931 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:38.047531 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 521/1044] 10.222.0.17 () {58 vars in 1058 bytes} [Mon Oct 17 03:43:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:43:38.055987 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f405f078-5b76-4789-82b9-92b4598aba16 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:43:38.057844 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-f405f078-5b76-4789-82b9-92b4598aba16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:38.212257 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f405f078-5b76-4789-82b9-92b4598aba16 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:43:38.234051 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:38.332034 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e02cc-c628-40a7-bad5-685d3228a046 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:43:38.332647 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e02cc-c628-40a7-bad5-685d3228a046 service nova] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:38.332921 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e02cc-c628-40a7-bad5-685d3228a046 service nova] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:38.333188 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e02cc-c628-40a7-bad5-685d3228a046 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:38.334782 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-0d3e02cc-c628-40a7-bad5-685d3228a046 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:38.434494 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e02cc-c628-40a7-bad5-685d3228a046 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:43:38.436107 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e02cc-c628-40a7-bad5-685d3228a046 service nova] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:43:38.454619 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 524/1045] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:43:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:43:38.475867 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78b551-ca3a-47e6-adca-0cfa686b5551 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:38.476375 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78b551-ca3a-47e6-adca-0cfa686b5551 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:38.476481 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f78b551-ca3a-47e6-adca-0cfa686b5551 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:38.478326 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 522/1046] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:43:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:43:38.483679 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603adf6-d71d-49df-92e0-3b7e8ac50502 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:38.484431 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603adf6-d71d-49df-92e0-3b7e8ac50502 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:38.484431 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603adf6-d71d-49df-92e0-3b7e8ac50502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:38.517116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:38.551878 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-a603adf6-d71d-49df-92e0-3b7e8ac50502 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:43:38.553530 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-a603adf6-d71d-49df-92e0-3b7e8ac50502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:38.667898 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 525/1047] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:43:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:43:39.005309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46392c8-1b95-40fa-a064-22ce6fccb92d None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:39.005309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46392c8-1b95-40fa-a064-22ce6fccb92d None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:39.005309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46392c8-1b95-40fa-a064-22ce6fccb92d None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:39.007500 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 523/1048] 10.222.0.17 () {54 vars in 963 bytes} [Mon Oct 17 03:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:43:39.012491 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1771ed-287f-42d8-aa17-5bfe8b02d0a8 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:39.012491 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1771ed-287f-42d8-aa17-5bfe8b02d0a8 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:39.012491 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1771ed-287f-42d8-aa17-5bfe8b02d0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:39.091708 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-af1771ed-287f-42d8-aa17-5bfe8b02d0a8 None None] MFA Rules not processed for user `99bc94105bbf4cb4a60e37133edc8dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:43:39.093718 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-af1771ed-287f-42d8-aa17-5bfe8b02d0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:39.144670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:39.175142 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 526/1049] 10.222.0.17 () {58 vars in 1060 bytes} [Mon Oct 17 03:43:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:43:39.183428 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46392c8-1b95-40fa-a064-22ce6fccb92d None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:43:39.185158 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-e46392c8-1b95-40fa-a064-22ce6fccb92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:39.295446 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46392c8-1b95-40fa-a064-22ce6fccb92d None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:43:39.337802 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:39.375709 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e0e8c4-c135-4721-983a-c58a7e6931b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:43:39.376353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0e8c4-c135-4721-983a-c58a7e6931b8 service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:39.376632 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0e8c4-c135-4721-983a-c58a7e6931b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:39.377101 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e0e8c4-c135-4721-983a-c58a7e6931b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:39.378894 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-06e0e8c4-c135-4721-983a-c58a7e6931b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:39.502934 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e0e8c4-c135-4721-983a-c58a7e6931b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:43:39.504473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e0e8c4-c135-4721-983a-c58a7e6931b8 service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:43:39.524396 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 524/1050] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:43:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:43:40.775367 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555370a-7265-4e56-9222-ba228e7d8f94 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:40.775996 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555370a-7265-4e56-9222-ba228e7d8f94 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:40.775996 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555370a-7265-4e56-9222-ba228e7d8f94 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:40.777394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 527/1051] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:43:40.782748 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ee5f0-4891-4e73-95b2-544bec1878fe None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:40.783153 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ee5f0-4891-4e73-95b2-544bec1878fe None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:40.783320 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ee5f0-4891-4e73-95b2-544bec1878fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:40.849326 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-1a5ee5f0-4891-4e73-95b2-544bec1878fe None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:43:40.850549 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-1a5ee5f0-4891-4e73-95b2-544bec1878fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:40.871895 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:40.949314 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 525/1052] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:43:41.242298 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555370a-7265-4e56-9222-ba228e7d8f94 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:43:41.243857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-5555370a-7265-4e56-9222-ba228e7d8f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:41.393306 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555370a-7265-4e56-9222-ba228e7d8f94 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:43:41.447377 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:41.512099 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924c23c-74c2-435a-8642-fb962e21d779 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:43:41.512719 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924c23c-74c2-435a-8642-fb962e21d779 service nova] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:41.512999 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924c23c-74c2-435a-8642-fb962e21d779 service nova] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:41.513275 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924c23c-74c2-435a-8642-fb962e21d779 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:41.514872 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-6924c23c-74c2-435a-8642-fb962e21d779 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:41.610570 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6924c23c-74c2-435a-8642-fb962e21d779 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:43:41.611825 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6924c23c-74c2-435a-8642-fb962e21d779 service nova] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:43:41.629968 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 528/1053] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:43:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:43:43.921452 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd8815-146f-4f9e-89a4-a03db70511c7 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:43.922021 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd8815-146f-4f9e-89a4-a03db70511c7 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:43.928496 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd8815-146f-4f9e-89a4-a03db70511c7 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:43.928496 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 526/1054] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:43:43.928914 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe70b9-f803-4cad-b15b-b7765477f7fd None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:43.929369 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe70b9-f803-4cad-b15b-b7765477f7fd None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:43.929687 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe70b9-f803-4cad-b15b-b7765477f7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:43.998918 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-9dbe70b9-f803-4cad-b15b-b7765477f7fd None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:43:44.000238 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-9dbe70b9-f803-4cad-b15b-b7765477f7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:44.008105 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:44.100689 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 529/1055] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:43:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:43:44.390347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd8815-146f-4f9e-89a4-a03db70511c7 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:43:44.392188 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-d5fd8815-146f-4f9e-89a4-a03db70511c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:44.548597 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd8815-146f-4f9e-89a4-a03db70511c7 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:43:44.553045 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:44.669764 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128bdcb7-69cb-4bd3-8b3a-a9886021b609 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:43:44.670296 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128bdcb7-69cb-4bd3-8b3a-a9886021b609 service nova] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:44.670562 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128bdcb7-69cb-4bd3-8b3a-a9886021b609 service nova] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:44.670831 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128bdcb7-69cb-4bd3-8b3a-a9886021b609 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:44.673149 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-128bdcb7-69cb-4bd3-8b3a-a9886021b609 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:44.774839 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128bdcb7-69cb-4bd3-8b3a-a9886021b609 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:43:44.776593 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128bdcb7-69cb-4bd3-8b3a-a9886021b609 service nova] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:43:44.797492 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 527/1056] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:43:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:43:47.087045 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e06aa-da94-42c4-b48f-eea2d59f241f None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:47.087685 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e06aa-da94-42c4-b48f-eea2d59f241f None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:47.087685 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e06aa-da94-42c4-b48f-eea2d59f241f None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:47.093704 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 530/1057] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:43:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:43:47.094441 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b0dc5-2bc0-4571-aa51-5fdfb5754b9d None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:47.095343 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b0dc5-2bc0-4571-aa51-5fdfb5754b9d None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:47.095343 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b0dc5-2bc0-4571-aa51-5fdfb5754b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:47.110424 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:47.200556 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-872b0dc5-2bc0-4571-aa51-5fdfb5754b9d None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:43:47.201782 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-872b0dc5-2bc0-4571-aa51-5fdfb5754b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:47.302229 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 528/1058] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:43:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:43:47.597949 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e06aa-da94-42c4-b48f-eea2d59f241f None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:43:47.599566 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-185e06aa-da94-42c4-b48f-eea2d59f241f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:47.744837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:47.755241 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e06aa-da94-42c4-b48f-eea2d59f241f None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:43:47.874696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3d334-9a52-453e-bdc2-6d3082fdbf8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:43:47.875231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3d334-9a52-453e-bdc2-6d3082fdbf8b service nova] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:47.875541 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3d334-9a52-453e-bdc2-6d3082fdbf8b service nova] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:47.875854 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3d334-9a52-453e-bdc2-6d3082fdbf8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:47.877460 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-12c3d334-9a52-453e-bdc2-6d3082fdbf8b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:47.970338 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3d334-9a52-453e-bdc2-6d3082fdbf8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:43:47.972630 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3d334-9a52-453e-bdc2-6d3082fdbf8b service nova] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:43:47.986254 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:47.995148 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 531/1059] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:43:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:43:50.660523 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f3786-d3be-4118-9b56-293e3977c015 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:50.661062 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f3786-d3be-4118-9b56-293e3977c015 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:50.661062 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f3786-d3be-4118-9b56-293e3977c015 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:50.662450 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 529/1060] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:43:50.668957 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcf6e7-56a8-46bb-b177-8c9e8d684dcf None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:50.669455 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcf6e7-56a8-46bb-b177-8c9e8d684dcf None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:50.669616 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dcf6e7-56a8-46bb-b177-8c9e8d684dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:50.738043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-88dcf6e7-56a8-46bb-b177-8c9e8d684dcf None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:43:50.739280 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-88dcf6e7-56a8-46bb-b177-8c9e8d684dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:50.837827 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 532/1061] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:43:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:43:51.130051 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f3786-d3be-4118-9b56-293e3977c015 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:43:51.132147 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-a25f3786-d3be-4118-9b56-293e3977c015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:51.244664 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:51.309046 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f3786-d3be-4118-9b56-293e3977c015 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:43:51.434655 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74578d4-0211-44ba-9d36-d4658f85ad92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:43:51.435185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74578d4-0211-44ba-9d36-d4658f85ad92 service nova] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:51.435462 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74578d4-0211-44ba-9d36-d4658f85ad92 service nova] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:51.435753 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74578d4-0211-44ba-9d36-d4658f85ad92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:51.437550 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-b74578d4-0211-44ba-9d36-d4658f85ad92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:51.489086 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:51.529105 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74578d4-0211-44ba-9d36-d4658f85ad92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:43:51.531316 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74578d4-0211-44ba-9d36-d4658f85ad92 service nova] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:43:51.555196 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 530/1062] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:43:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:43:53.709203 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aec64a-80a3-4c8e-8cae-f18af22ee753 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:53.709710 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aec64a-80a3-4c8e-8cae-f18af22ee753 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:53.709824 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aec64a-80a3-4c8e-8cae-f18af22ee753 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:53.711957 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 533/1063] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:43:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:43:53.717153 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ad526-1219-4c9f-8242-9459c9875b83 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:53.717555 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ad526-1219-4c9f-8242-9459c9875b83 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:53.717727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ad526-1219-4c9f-8242-9459c9875b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:53.783249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-f79ad526-1219-4c9f-8242-9459c9875b83 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:43:53.784503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-f79ad526-1219-4c9f-8242-9459c9875b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:53.880867 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 531/1064] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:43:54.168915 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aec64a-80a3-4c8e-8cae-f18af22ee753 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:43:54.170497 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-16aec64a-80a3-4c8e-8cae-f18af22ee753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:54.247334 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:54.315229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aec64a-80a3-4c8e-8cae-f18af22ee753 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:43:54.425568 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377426d1-26b3-4396-9e01-a0081cd35eb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:43:54.426085 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377426d1-26b3-4396-9e01-a0081cd35eb9 service nova] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:54.426361 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377426d1-26b3-4396-9e01-a0081cd35eb9 service nova] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:54.426627 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377426d1-26b3-4396-9e01-a0081cd35eb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:54.428237 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-377426d1-26b3-4396-9e01-a0081cd35eb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:54.467185 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:54.518362 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377426d1-26b3-4396-9e01-a0081cd35eb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:43:54.519958 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377426d1-26b3-4396-9e01-a0081cd35eb9 service nova] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:43:54.537671 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 534/1065] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:43:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:43:54.941216 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908463fc-dfd6-4c23-a666-603f89604ffa None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:54.941740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908463fc-dfd6-4c23-a666-603f89604ffa None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:54.941740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908463fc-dfd6-4c23-a666-603f89604ffa None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:54.943033 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 532/1066] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:43:54.949108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c3d09-061b-4f3b-9623-a963de2f9098 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:54.949534 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c3d09-061b-4f3b-9623-a963de2f9098 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:54.949707 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c3d09-061b-4f3b-9623-a963de2f9098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:55.017384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-ea3c3d09-061b-4f3b-9623-a963de2f9098 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:43:55.018599 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-ea3c3d09-061b-4f3b-9623-a963de2f9098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:55.118082 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 535/1067] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:43:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:43:55.417288 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb1974-1e8d-4cf7-a218-041a861ea78a None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:55.417654 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb1974-1e8d-4cf7-a218-041a861ea78a None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:55.417999 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb1974-1e8d-4cf7-a218-041a861ea78a None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:55.419613 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 533/1068] 10.222.0.17 () {54 vars in 962 bytes} [Mon Oct 17 03:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:43:55.424864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be7ea8a-0ac3-400f-9c5c-cc784ca939ad None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:55.425151 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be7ea8a-0ac3-400f-9c5c-cc784ca939ad None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:55.425447 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be7ea8a-0ac3-400f-9c5c-cc784ca939ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:55.441434 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:55.501938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-1be7ea8a-0ac3-400f-9c5c-cc784ca939ad None None] MFA Rules not processed for user `565a836c7f89478cad615bd6c14531a7`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:43:55.503063 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-1be7ea8a-0ac3-400f-9c5c-cc784ca939ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:55.579687 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 536/1069] 10.222.0.17 () {58 vars in 1059 bytes} [Mon Oct 17 03:43:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:43:55.587586 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb1974-1e8d-4cf7-a218-041a861ea78a None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:43:55.589164 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-52eb1974-1e8d-4cf7-a218-041a861ea78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:55.621777 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:55.694676 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb1974-1e8d-4cf7-a218-041a861ea78a None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:43:55.771905 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10520835-da4a-4cc3-8678-8f8b4a4de013 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '565a836c7f89478cad615bd6c14531a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:43:55.772551 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10520835-da4a-4cc3-8678-8f8b4a4de013 service glance] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:55.772835 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10520835-da4a-4cc3-8678-8f8b4a4de013 service glance] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:55.773109 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10520835-da4a-4cc3-8678-8f8b4a4de013 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:55.774728 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-10520835-da4a-4cc3-8678-8f8b4a4de013 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:55.852285 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:55.884336 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10520835-da4a-4cc3-8678-8f8b4a4de013 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:43:55.886098 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10520835-da4a-4cc3-8678-8f8b4a4de013 service glance] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:43:55.903933 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 534/1070] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:43:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:43:56.823726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab846fa-e323-4cdb-bc37-4b6937af1b3c None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:56.824227 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab846fa-e323-4cdb-bc37-4b6937af1b3c None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:56.824319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab846fa-e323-4cdb-bc37-4b6937af1b3c None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:56.825702 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 537/1071] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:43:56.827567 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c746fb5-ec19-4d23-9e30-1cbc61572acc None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:56.828002 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c746fb5-ec19-4d23-9e30-1cbc61572acc None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:56.828189 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c746fb5-ec19-4d23-9e30-1cbc61572acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:56.830874 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1bdb9a-41e8-49e8-9abe-dbe22560005b None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:56.831219 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1bdb9a-41e8-49e8-9abe-dbe22560005b None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:56.831561 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1bdb9a-41e8-49e8-9abe-dbe22560005b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:56.895231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-6c746fb5-ec19-4d23-9e30-1cbc61572acc None None] MFA Rules not processed for user `cecbc19355ef4570b2b05bba876a3a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:43:56.896593 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6c746fb5-ec19-4d23-9e30-1cbc61572acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:56.899944 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-7d1bdb9a-41e8-49e8-9abe-dbe22560005b None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:43:56.901147 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-7d1bdb9a-41e8-49e8-9abe-dbe22560005b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:56.957152 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:56.986687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 535/1072] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Oct 17 03:43:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:43:57.000783 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 538/1073] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:43:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:43:57.034380 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86a226-cb1f-49b8-b0e7-68c21f8a36ab None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:57.034761 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86a226-cb1f-49b8-b0e7-68c21f8a36ab None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:57.035108 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86a226-cb1f-49b8-b0e7-68c21f8a36ab None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:57.036575 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 536/1074] 10.222.0.17 () {54 vars in 943 bytes} [Mon Oct 17 03:43:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:43:57.043080 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36213b4-066c-4e0e-a3a8-f44caf0c73f1 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:57.043370 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36213b4-066c-4e0e-a3a8-f44caf0c73f1 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:57.043646 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36213b4-066c-4e0e-a3a8-f44caf0c73f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:57.120000 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-d36213b4-066c-4e0e-a3a8-f44caf0c73f1 None None] MFA Rules not processed for user `5f64aaf4d408462a88963973d42abfb9`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:43:57.121219 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-d36213b4-066c-4e0e-a3a8-f44caf0c73f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:57.224432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 539/1075] 10.222.0.17 () {58 vars in 1040 bytes} [Mon Oct 17 03:43:57 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:43:57.234290 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86a226-cb1f-49b8-b0e7-68c21f8a36ab None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:43:57.235863 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-da86a226-cb1f-49b8-b0e7-68c21f8a36ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:57.286841 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:57.291719 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36213b4-066c-4e0e-a3a8-f44caf0c73f1 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:43:57.292942 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-d36213b4-066c-4e0e-a3a8-f44caf0c73f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:57.314441 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:57.375181 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da86a226-cb1f-49b8-b0e7-68c21f8a36ab None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:43:57.392421 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36213b4-066c-4e0e-a3a8-f44caf0c73f1 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:43:57.480651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63833730-e8fa-489e-8970-223d40a4757a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f64aaf4d408462a88963973d42abfb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:43:57.481183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63833730-e8fa-489e-8970-223d40a4757a service swift] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:57.481467 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63833730-e8fa-489e-8970-223d40a4757a service swift] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:57.481740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63833730-e8fa-489e-8970-223d40a4757a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:57.483415 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-63833730-e8fa-489e-8970-223d40a4757a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:57.507306 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c19b28-bdb1-4a10-be1f-0718a7742677 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:43:57.507955 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c19b28-bdb1-4a10-be1f-0718a7742677 service nova] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:43:57.508244 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c19b28-bdb1-4a10-be1f-0718a7742677 service nova] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:43:57.508520 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c19b28-bdb1-4a10-be1f-0718a7742677 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:43:57.510759 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-75c19b28-bdb1-4a10-be1f-0718a7742677 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:43:57.517085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:57.542001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:43:57.570637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63833730-e8fa-489e-8970-223d40a4757a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:43:57.572288 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63833730-e8fa-489e-8970-223d40a4757a service swift] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:43:57.580369 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 537/1076] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Oct 17 03:43:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 17 03:43:57.603677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c19b28-bdb1-4a10-be1f-0718a7742677 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:43:57.605289 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c19b28-bdb1-4a10-be1f-0718a7742677 service nova] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:43:57.623320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 540/1077] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:43:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:44:00.132592 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6448a-68ce-45c7-acf2-bffe1686407a None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:00.133200 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6448a-68ce-45c7-acf2-bffe1686407a None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:00.133200 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6448a-68ce-45c7-acf2-bffe1686407a None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:00.134472 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 538/1078] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:44:00.139847 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190f66a-a3a4-42e0-9f2f-681cda23d84f None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:00.140704 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190f66a-a3a4-42e0-9f2f-681cda23d84f None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:00.141046 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d190f66a-a3a4-42e0-9f2f-681cda23d84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:00.216987 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-d190f66a-a3a4-42e0-9f2f-681cda23d84f None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:44:00.222391 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-d190f66a-a3a4-42e0-9f2f-681cda23d84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:00.330420 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 541/1079] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:44:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:44:00.633468 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa6448a-68ce-45c7-acf2-bffe1686407a None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:44:00.634991 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-2fa6448a-68ce-45c7-acf2-bffe1686407a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:00.797600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa6448a-68ce-45c7-acf2-bffe1686407a None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:44:00.830910 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:00.933647 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4882058b-96ce-4078-92ba-bfca1ba9260c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:44:00.934440 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882058b-96ce-4078-92ba-bfca1ba9260c service nova] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:00.934816 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882058b-96ce-4078-92ba-bfca1ba9260c service nova] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:00.935157 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4882058b-96ce-4078-92ba-bfca1ba9260c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:00.937214 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-4882058b-96ce-4078-92ba-bfca1ba9260c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:01.044869 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882058b-96ce-4078-92ba-bfca1ba9260c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:44:01.044869 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4882058b-96ce-4078-92ba-bfca1ba9260c service nova] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:44:01.070596 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 539/1080] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:44:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:44:03.432654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545dad7d-c446-4207-ab5d-edd55d4a753d None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:03.433204 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545dad7d-c446-4207-ab5d-edd55d4a753d None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:03.433204 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545dad7d-c446-4207-ab5d-edd55d4a753d None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:03.434557 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 542/1081] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:44:03.440112 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f5487-b7ca-455f-b99d-bd5bea392457 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:03.441012 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f5487-b7ca-455f-b99d-bd5bea392457 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:03.441477 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f5487-b7ca-455f-b99d-bd5bea392457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:03.485257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:03.511449 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-1b5f5487-b7ca-455f-b99d-bd5bea392457 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:44:03.513033 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-1b5f5487-b7ca-455f-b99d-bd5bea392457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:03.622049 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 540/1082] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:44:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:44:03.914350 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545dad7d-c446-4207-ab5d-edd55d4a753d None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:44:03.915911 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-545dad7d-c446-4207-ab5d-edd55d4a753d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:03.925002 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:04.081713 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545dad7d-c446-4207-ab5d-edd55d4a753d None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:44:04.168644 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:04.204207 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443c7a3-8af7-4b50-9096-374c7ee31213 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:44:04.204823 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443c7a3-8af7-4b50-9096-374c7ee31213 service nova] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:04.205106 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443c7a3-8af7-4b50-9096-374c7ee31213 service nova] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:04.205387 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443c7a3-8af7-4b50-9096-374c7ee31213 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:04.207147 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-7443c7a3-8af7-4b50-9096-374c7ee31213 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:04.311956 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443c7a3-8af7-4b50-9096-374c7ee31213 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:44:04.313755 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443c7a3-8af7-4b50-9096-374c7ee31213 service nova] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:44:04.336084 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 543/1083] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:44:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:44:06.745084 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb5d59-d374-404f-b1cb-2b780d7599bf None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:06.745642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb5d59-d374-404f-b1cb-2b780d7599bf None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:06.745642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdb5d59-d374-404f-b1cb-2b780d7599bf None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:06.746972 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 541/1084] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:44:06.752261 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdd59b-dc18-47df-a185-3a59d6ea10e7 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:06.752681 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdd59b-dc18-47df-a185-3a59d6ea10e7 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:06.753360 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcdd59b-dc18-47df-a185-3a59d6ea10e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:06.831718 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-1bcdd59b-dc18-47df-a185-3a59d6ea10e7 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:44:06.833308 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-1bcdd59b-dc18-47df-a185-3a59d6ea10e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:06.882997 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:06.947888 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 544/1085] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:44:07.235349 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdb5d59-d374-404f-b1cb-2b780d7599bf None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:44:07.236915 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-dfdb5d59-d374-404f-b1cb-2b780d7599bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:07.358567 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:07.402486 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdb5d59-d374-404f-b1cb-2b780d7599bf None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:44:07.530033 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0b087d-93b6-4637-9669-26102f160fe8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:44:07.530567 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b087d-93b6-4637-9669-26102f160fe8 service nova] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:07.530850 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b087d-93b6-4637-9669-26102f160fe8 service nova] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:07.531122 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0b087d-93b6-4637-9669-26102f160fe8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:07.532842 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-4c0b087d-93b6-4637-9669-26102f160fe8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:07.614839 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:07.631138 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b087d-93b6-4637-9669-26102f160fe8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:44:07.632862 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0b087d-93b6-4637-9669-26102f160fe8 service nova] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:44:07.652396 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 542/1086] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:44:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:44:16.959887 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=24 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:17.650051 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=13 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:21.341600 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576fb805-d84d-45df-868b-8494b90e7616 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:21.342127 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576fb805-d84d-45df-868b-8494b90e7616 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:21.342127 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576fb805-d84d-45df-868b-8494b90e7616 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:21.343470 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 545/1087] 10.222.0.17 () {54 vars in 918 bytes} [Mon Oct 17 03:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:44:21.348673 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af7b05-88f9-4c51-b0e7-95e1ccdcef6b None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:21.349590 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af7b05-88f9-4c51-b0e7-95e1ccdcef6b None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:21.349752 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af7b05-88f9-4c51-b0e7-95e1ccdcef6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:21.423466 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-b4af7b05-88f9-4c51-b0e7-95e1ccdcef6b None None] MFA Rules not processed for user `b500a113a2aa4a95875d7e094b917db3`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:44:21.424766 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-b4af7b05-88f9-4c51-b0e7-95e1ccdcef6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:21.528173 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 543/1088] 10.222.0.17 () {58 vars in 1015 bytes} [Mon Oct 17 03:44:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:44:21.543089 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af3e9d-a37f-4d44-9a47-cd6fc5a35e1e None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:21.543413 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af3e9d-a37f-4d44-9a47-cd6fc5a35e1e None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:21.543687 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98af3e9d-a37f-4d44-9a47-cd6fc5a35e1e None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:21.545028 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 546/1089] 10.222.0.17 () {54 vars in 918 bytes} [Mon Oct 17 03:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:44:21.549622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6396b-a5ad-4f83-95ae-5eb155445167 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:21.549904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6396b-a5ad-4f83-95ae-5eb155445167 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:21.550178 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6396b-a5ad-4f83-95ae-5eb155445167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:21.623560 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:21.626076 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-f8d6396b-a5ad-4f83-95ae-5eb155445167 None None] MFA Rules not processed for user `d961b751763a4e49b5d4ca4c795c5bf6`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:44:21.627275 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-f8d6396b-a5ad-4f83-95ae-5eb155445167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:21.741862 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 544/1090] 10.222.0.17 () {58 vars in 1015 bytes} [Mon Oct 17 03:44:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:44:24.290241 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91086825-bd6e-40de-a6a9-3c0311f19c41 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:24.294926 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91086825-bd6e-40de-a6a9-3c0311f19c41 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:24.294926 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91086825-bd6e-40de-a6a9-3c0311f19c41 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:24.294926 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 547/1091] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:44:24.297604 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136bade-b4ed-4876-b246-fe33b96fee92 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:24.298130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136bade-b4ed-4876-b246-fe33b96fee92 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:24.298246 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5136bade-b4ed-4876-b246-fe33b96fee92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:24.365740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-5136bade-b4ed-4876-b246-fe33b96fee92 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:44:24.366941 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-5136bade-b4ed-4876-b246-fe33b96fee92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:24.419962 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:24.461722 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 545/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:44:24.752646 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91086825-bd6e-40de-a6a9-3c0311f19c41 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:44:24.754158 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-91086825-bd6e-40de-a6a9-3c0311f19c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:24.894065 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91086825-bd6e-40de-a6a9-3c0311f19c41 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:44:24.965385 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:25.003857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b72e61-64ab-492e-a433-6f76f3aded77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:44:25.004460 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b72e61-64ab-492e-a433-6f76f3aded77 service nova] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:25.004737 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b72e61-64ab-492e-a433-6f76f3aded77 service nova] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:25.005004 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b72e61-64ab-492e-a433-6f76f3aded77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:25.007123 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-51b72e61-64ab-492e-a433-6f76f3aded77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:25.094220 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b72e61-64ab-492e-a433-6f76f3aded77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:44:25.095827 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b72e61-64ab-492e-a433-6f76f3aded77 service nova] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:44:25.113094 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 548/1093] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:44:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:44:26.236568 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5522a1e-6abf-48e5-ba77-29e0de197f18 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:26.237108 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5522a1e-6abf-48e5-ba77-29e0de197f18 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:26.237108 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5522a1e-6abf-48e5-ba77-29e0de197f18 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:26.238456 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 546/1094] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 03:44:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:44:26.244730 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74123c3d-6cfc-4b57-aedf-440f542dbdca None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:26.245166 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74123c3d-6cfc-4b57-aedf-440f542dbdca None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:26.245289 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74123c3d-6cfc-4b57-aedf-440f542dbdca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:26.320569 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-74123c3d-6cfc-4b57-aedf-440f542dbdca None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:44:26.321775 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-74123c3d-6cfc-4b57-aedf-440f542dbdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:26.348246 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:26.415920 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 549/1095] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 03:44:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:44:26.564067 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5522a1e-6abf-48e5-ba77-29e0de197f18 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:44:26.565624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-a5522a1e-6abf-48e5-ba77-29e0de197f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:26.717217 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5522a1e-6abf-48e5-ba77-29e0de197f18 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:44:26.757617 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:26.830578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7319ad-939e-4199-a9bc-f385e1292e60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:44:26.831130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7319ad-939e-4199-a9bc-f385e1292e60 service nova] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:26.831393 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7319ad-939e-4199-a9bc-f385e1292e60 service nova] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:26.831660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7319ad-939e-4199-a9bc-f385e1292e60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:26.833313 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-5a7319ad-939e-4199-a9bc-f385e1292e60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:26.921544 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7319ad-939e-4199-a9bc-f385e1292e60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:44:26.927705 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7319ad-939e-4199-a9bc-f385e1292e60 service nova] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:44:26.941678 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 547/1096] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:44:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:44:28.071611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8256f0b-1fd8-4fca-beed-a23e6b2c4dc5 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:28.072212 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8256f0b-1fd8-4fca-beed-a23e6b2c4dc5 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:28.072212 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8256f0b-1fd8-4fca-beed-a23e6b2c4dc5 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:28.073528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 550/1097] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 03:44:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:44:28.079588 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681f797-9a33-4820-b0bd-40934c94b776 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:28.080016 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681f797-9a33-4820-b0bd-40934c94b776 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:28.080174 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681f797-9a33-4820-b0bd-40934c94b776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:28.133916 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:28.155187 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-6681f797-9a33-4820-b0bd-40934c94b776 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:44:28.156445 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6681f797-9a33-4820-b0bd-40934c94b776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:28.252250 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 548/1098] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 03:44:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:44:28.401573 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8256f0b-1fd8-4fca-beed-a23e6b2c4dc5 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:44:28.403089 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-e8256f0b-1fd8-4fca-beed-a23e6b2c4dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:28.546532 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8256f0b-1fd8-4fca-beed-a23e6b2c4dc5 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:44:28.558439 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:28.656529 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a314e80d-3583-4898-9f50-b3d5f5d83342 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:44:28.657060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314e80d-3583-4898-9f50-b3d5f5d83342 service nova] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:28.657340 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314e80d-3583-4898-9f50-b3d5f5d83342 service nova] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:28.657612 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a314e80d-3583-4898-9f50-b3d5f5d83342 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:28.659231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-a314e80d-3583-4898-9f50-b3d5f5d83342 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:28.747966 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a314e80d-3583-4898-9f50-b3d5f5d83342 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:44:28.749537 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a314e80d-3583-4898-9f50-b3d5f5d83342 service nova] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:44:28.767646 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 551/1099] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:44:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:44:29.870501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226c7b4-6c58-4b0b-bbe2-ef6d449b20e1 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:29.871071 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226c7b4-6c58-4b0b-bbe2-ef6d449b20e1 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:29.871071 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226c7b4-6c58-4b0b-bbe2-ef6d449b20e1 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:29.872418 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 549/1100] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 03:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:44:29.878723 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8475ad5-7406-44f5-9f0d-c8306c3e4fb0 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:29.879189 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8475ad5-7406-44f5-9f0d-c8306c3e4fb0 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:29.879302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8475ad5-7406-44f5-9f0d-c8306c3e4fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:29.899027 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:29.955884 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-e8475ad5-7406-44f5-9f0d-c8306c3e4fb0 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:44:29.957014 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-e8475ad5-7406-44f5-9f0d-c8306c3e4fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:30.050515 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 552/1101] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 03:44:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:44:30.199574 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9226c7b4-6c58-4b0b-bbe2-ef6d449b20e1 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:44:30.201154 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-9226c7b4-6c58-4b0b-bbe2-ef6d449b20e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:30.309937 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:30.346313 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9226c7b4-6c58-4b0b-bbe2-ef6d449b20e1 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:44:30.456378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ce66e-dd84-48a4-b0c0-6a2c136aa4ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:44:30.456978 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ce66e-dd84-48a4-b0c0-6a2c136aa4ba service nova] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:30.457253 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ce66e-dd84-48a4-b0c0-6a2c136aa4ba service nova] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:30.457525 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ce66e-dd84-48a4-b0c0-6a2c136aa4ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:30.459727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-f79ce66e-dd84-48a4-b0c0-6a2c136aa4ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:30.533997 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:30.546920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79ce66e-dd84-48a4-b0c0-6a2c136aa4ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:44:30.548555 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79ce66e-dd84-48a4-b0c0-6a2c136aa4ba service nova] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:44:30.566601 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 550/1102] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:44:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:44:31.663696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a35562-b655-422a-bd96-34cfe669c835 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:31.664314 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a35562-b655-422a-bd96-34cfe669c835 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:31.664439 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a35562-b655-422a-bd96-34cfe669c835 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:31.665701 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 553/1103] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 03:44:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:44:31.671763 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec3c63-b9d9-4f37-a5ff-4e09d77114c4 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:31.672218 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec3c63-b9d9-4f37-a5ff-4e09d77114c4 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:31.672326 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ec3c63-b9d9-4f37-a5ff-4e09d77114c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:31.749415 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-a1ec3c63-b9d9-4f37-a5ff-4e09d77114c4 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:44:31.750620 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-a1ec3c63-b9d9-4f37-a5ff-4e09d77114c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:31.846096 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 551/1104] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 03:44:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:44:31.996194 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a35562-b655-422a-bd96-34cfe669c835 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:44:31.997700 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-b3a35562-b655-422a-bd96-34cfe669c835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:32.079164 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:32.140613 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a35562-b655-422a-bd96-34cfe669c835 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:44:32.251352 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba2c69-f92d-414c-b01c-3b9eec77be7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:44:32.251979 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba2c69-f92d-414c-b01c-3b9eec77be7c service nova] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:32.252254 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba2c69-f92d-414c-b01c-3b9eec77be7c service nova] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:32.252519 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba2c69-f92d-414c-b01c-3b9eec77be7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:32.254095 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-dcba2c69-f92d-414c-b01c-3b9eec77be7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:32.302092 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:32.342309 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba2c69-f92d-414c-b01c-3b9eec77be7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:44:32.343892 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba2c69-f92d-414c-b01c-3b9eec77be7c service nova] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:44:32.361842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 554/1105] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:44:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:44:33.458692 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f5e85-cc5a-4912-9cda-7a4dafd32b45 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:33.459263 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f5e85-cc5a-4912-9cda-7a4dafd32b45 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:33.459263 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f5e85-cc5a-4912-9cda-7a4dafd32b45 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:33.460631 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 552/1106] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 03:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:44:33.466894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdab628-15e6-4866-839e-90e34dfa1a5f None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:33.467300 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdab628-15e6-4866-839e-90e34dfa1a5f None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:33.467440 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdab628-15e6-4866-839e-90e34dfa1a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:33.542338 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-ebdab628-15e6-4866-839e-90e34dfa1a5f None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:44:33.543446 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-ebdab628-15e6-4866-839e-90e34dfa1a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:33.636783 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 555/1107] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 03:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:44:33.784989 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f5e85-cc5a-4912-9cda-7a4dafd32b45 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:44:33.787165 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-158f5e85-cc5a-4912-9cda-7a4dafd32b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:33.835091 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:33.930026 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f5e85-cc5a-4912-9cda-7a4dafd32b45 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:44:34.043144 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a16bd88-2222-4b05-b261-3c5f2ffde7de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:44:34.043774 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16bd88-2222-4b05-b261-3c5f2ffde7de service nova] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:34.044055 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16bd88-2222-4b05-b261-3c5f2ffde7de service nova] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:34.044604 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a16bd88-2222-4b05-b261-3c5f2ffde7de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:34.046225 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-0a16bd88-2222-4b05-b261-3c5f2ffde7de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:34.053156 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:34.135150 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a16bd88-2222-4b05-b261-3c5f2ffde7de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:44:34.136699 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a16bd88-2222-4b05-b261-3c5f2ffde7de service nova] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:44:34.155447 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 553/1108] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:44:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:44:35.256658 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf78775-8b15-4a6f-b565-e909148f8f0a None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:35.257076 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf78775-8b15-4a6f-b565-e909148f8f0a None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:35.257268 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf78775-8b15-4a6f-b565-e909148f8f0a None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:35.258524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 556/1109] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 03:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:44:35.264824 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85b2d9-a808-4ac6-938e-d9bf4060002e None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:35.265265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85b2d9-a808-4ac6-938e-d9bf4060002e None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:35.265370 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85b2d9-a808-4ac6-938e-d9bf4060002e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:35.339560 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-2e85b2d9-a808-4ac6-938e-d9bf4060002e None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:44:35.341042 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-2e85b2d9-a808-4ac6-938e-d9bf4060002e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:35.407744 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:35.437564 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 554/1110] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 03:44:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:44:35.589433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf78775-8b15-4a6f-b565-e909148f8f0a None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:44:35.591184 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-4bf78775-8b15-4a6f-b565-e909148f8f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:35.608607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:35.733315 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf78775-8b15-4a6f-b565-e909148f8f0a None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:44:35.818402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:35.843160 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a7206-3c4a-4462-8f53-f464829c7acd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:44:35.843697 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a7206-3c4a-4462-8f53-f464829c7acd service nova] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:35.844007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a7206-3c4a-4462-8f53-f464829c7acd service nova] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:35.844272 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a7206-3c4a-4462-8f53-f464829c7acd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:35.845882 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-462a7206-3c4a-4462-8f53-f464829c7acd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:35.933908 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462a7206-3c4a-4462-8f53-f464829c7acd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:44:35.935560 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462a7206-3c4a-4462-8f53-f464829c7acd service nova] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:44:35.953914 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 557/1111] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:44:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:44:37.044929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea73f9c-9a3e-436e-bbfe-73b8b35675d4 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:37.045501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea73f9c-9a3e-436e-bbfe-73b8b35675d4 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:37.045501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea73f9c-9a3e-436e-bbfe-73b8b35675d4 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:37.046930 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 555/1112] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 03:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:44:37.053647 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7215b-db33-47fe-8534-fb97d219558a None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:37.054137 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7215b-db33-47fe-8534-fb97d219558a None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:37.054232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde7215b-db33-47fe-8534-fb97d219558a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:37.129864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-bde7215b-db33-47fe-8534-fb97d219558a None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:44:37.131053 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-bde7215b-db33-47fe-8534-fb97d219558a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:37.172374 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:37.225530 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 558/1113] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 03:44:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:44:37.375058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea73f9c-9a3e-436e-bbfe-73b8b35675d4 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:44:37.376563 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-7ea73f9c-9a3e-436e-bbfe-73b8b35675d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:37.520952 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea73f9c-9a3e-436e-bbfe-73b8b35675d4 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:44:37.574371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:37.634358 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575a896-a1d4-4355-95fa-279ea47e608d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:44:37.634889 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575a896-a1d4-4355-95fa-279ea47e608d service nova] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:37.635171 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575a896-a1d4-4355-95fa-279ea47e608d service nova] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:37.635446 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575a896-a1d4-4355-95fa-279ea47e608d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:37.637089 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6575a896-a1d4-4355-95fa-279ea47e608d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:37.726170 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575a896-a1d4-4355-95fa-279ea47e608d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:44:37.727788 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575a896-a1d4-4355-95fa-279ea47e608d service nova] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:44:37.745197 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 556/1114] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:44:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:44:38.828262 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2c023-b945-444c-b783-f28352efb189 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:38.828825 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2c023-b945-444c-b783-f28352efb189 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:38.828825 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2c023-b945-444c-b783-f28352efb189 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:38.830159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 559/1115] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 03:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:44:38.836876 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6ec59-9e08-4085-928d-96282dc7c9e4 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:38.837331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6ec59-9e08-4085-928d-96282dc7c9e4 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:38.837428 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6ec59-9e08-4085-928d-96282dc7c9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:38.913079 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-56e6ec59-9e08-4085-928d-96282dc7c9e4 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:44:38.914285 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-56e6ec59-9e08-4085-928d-96282dc7c9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:38.916355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:39.011056 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 557/1116] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 03:44:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:44:39.158931 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2c023-b945-444c-b783-f28352efb189 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:44:39.160542 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-b6d2c023-b945-444c-b783-f28352efb189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:39.305032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2c023-b945-444c-b783-f28352efb189 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:44:39.331706 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:39.415267 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcf2b0d-617a-4374-9ae5-c89ffa9e65c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:44:39.415892 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf2b0d-617a-4374-9ae5-c89ffa9e65c5 service nova] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:39.416170 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf2b0d-617a-4374-9ae5-c89ffa9e65c5 service nova] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:39.416438 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcf2b0d-617a-4374-9ae5-c89ffa9e65c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:39.418007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-9dcf2b0d-617a-4374-9ae5-c89ffa9e65c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:39.507723 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcf2b0d-617a-4374-9ae5-c89ffa9e65c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:44:39.509319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcf2b0d-617a-4374-9ae5-c89ffa9e65c5 service nova] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:44:39.527284 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 560/1117] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:44:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:44:41.585233 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bb58c-5c18-4c2f-9215-9ca4c1140f46 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:41.585779 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bb58c-5c18-4c2f-9215-9ca4c1140f46 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:41.585891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bb58c-5c18-4c2f-9215-9ca4c1140f46 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:41.587216 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 558/1118] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:44:41.592828 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e1dda-ac63-4bce-8dda-d5c146a365fa None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:41.593279 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e1dda-ac63-4bce-8dda-d5c146a365fa None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:41.593510 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e1dda-ac63-4bce-8dda-d5c146a365fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:41.634398 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:41.658836 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-902e1dda-ac63-4bce-8dda-d5c146a365fa None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:44:41.660069 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-902e1dda-ac63-4bce-8dda-d5c146a365fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:41.755716 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 561/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:44:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:44:42.043541 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bb58c-5c18-4c2f-9215-9ca4c1140f46 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:44:42.045151 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-4a9bb58c-5c18-4c2f-9215-9ca4c1140f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:42.149581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bb58c-5c18-4c2f-9215-9ca4c1140f46 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:44:42.182521 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:42.225398 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd8abf-ec06-41cd-b3aa-ff601075376e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:44:42.226033 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd8abf-ec06-41cd-b3aa-ff601075376e service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:42.226317 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd8abf-ec06-41cd-b3aa-ff601075376e service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:42.226592 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd8abf-ec06-41cd-b3aa-ff601075376e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:42.228238 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-79fd8abf-ec06-41cd-b3aa-ff601075376e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:42.335929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fd8abf-ec06-41cd-b3aa-ff601075376e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:44:42.337317 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fd8abf-ec06-41cd-b3aa-ff601075376e service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:44:42.355368 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 559/1120] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:44:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:44:44.656882 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de974fe9-8e02-4f22-acdb-c46da1624669 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:44.657461 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de974fe9-8e02-4f22-acdb-c46da1624669 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:44.657461 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de974fe9-8e02-4f22-acdb-c46da1624669 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:44.658755 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 562/1121] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:44:44.664493 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a642c-0cee-40e1-a2a8-9ff3e3103e18 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:44.664900 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a642c-0cee-40e1-a2a8-9ff3e3103e18 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:44.665055 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56a642c-0cee-40e1-a2a8-9ff3e3103e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:44.730737 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-d56a642c-0cee-40e1-a2a8-9ff3e3103e18 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:44:44.731974 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-d56a642c-0cee-40e1-a2a8-9ff3e3103e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:44.745165 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:44.828285 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 560/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:44:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:44:45.116354 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de974fe9-8e02-4f22-acdb-c46da1624669 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:44:45.117887 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-de974fe9-8e02-4f22-acdb-c46da1624669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:45.219345 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de974fe9-8e02-4f22-acdb-c46da1624669 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:44:45.231956 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:45.291508 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed4fe53-bf88-4903-92f3-ca0c8cdcecf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:44:45.292138 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4fe53-bf88-4903-92f3-ca0c8cdcecf5 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:45.292415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4fe53-bf88-4903-92f3-ca0c8cdcecf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:45.292681 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4fe53-bf88-4903-92f3-ca0c8cdcecf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:45.294261 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-9ed4fe53-bf88-4903-92f3-ca0c8cdcecf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:45.401160 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed4fe53-bf88-4903-92f3-ca0c8cdcecf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:44:45.402582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed4fe53-bf88-4903-92f3-ca0c8cdcecf5 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:44:45.421618 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 563/1123] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:44:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:44:47.906265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fead3f-6db6-4960-8f41-fe1769242755 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:47.906791 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fead3f-6db6-4960-8f41-fe1769242755 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:47.906791 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fead3f-6db6-4960-8f41-fe1769242755 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:47.908200 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 561/1124] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:44:47.914315 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e870d-f394-4a80-987f-399256a09b2a None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:47.914748 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e870d-f394-4a80-987f-399256a09b2a None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:47.914855 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e870d-f394-4a80-987f-399256a09b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:47.961171 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:47.980308 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-5f6e870d-f394-4a80-987f-399256a09b2a None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:44:47.981556 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-5f6e870d-f394-4a80-987f-399256a09b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:48.076068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 564/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:44:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:44:48.361833 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fead3f-6db6-4960-8f41-fe1769242755 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:44:48.363420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-64fead3f-6db6-4960-8f41-fe1769242755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:48.465415 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fead3f-6db6-4960-8f41-fe1769242755 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:44:48.504974 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:48.539132 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0dffdc-49a5-4986-9ee6-a6703a561b32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:44:48.539745 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0dffdc-49a5-4986-9ee6-a6703a561b32 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:48.540032 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0dffdc-49a5-4986-9ee6-a6703a561b32 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:48.540303 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0dffdc-49a5-4986-9ee6-a6703a561b32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:48.541897 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-2a0dffdc-49a5-4986-9ee6-a6703a561b32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:48.649289 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0dffdc-49a5-4986-9ee6-a6703a561b32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:44:48.650631 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0dffdc-49a5-4986-9ee6-a6703a561b32 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:44:48.668516 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 562/1126] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:44:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:44:50.925850 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101dfba-2796-4428-b0dc-77b2709032ca None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:50.926240 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101dfba-2796-4428-b0dc-77b2709032ca None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:50.926345 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101dfba-2796-4428-b0dc-77b2709032ca None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:50.927673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 565/1127] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:44:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:44:50.933164 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd56462-95a1-4872-9515-31464717c196 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:50.933564 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd56462-95a1-4872-9515-31464717c196 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:50.933668 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd56462-95a1-4872-9515-31464717c196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:50.999835 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-0bd56462-95a1-4872-9515-31464717c196 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:44:51.001033 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-0bd56462-95a1-4872-9515-31464717c196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:51.022056 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:51.098757 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 563/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:44:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:44:51.385373 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101dfba-2796-4428-b0dc-77b2709032ca None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:44:51.386934 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-9101dfba-2796-4428-b0dc-77b2709032ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:51.490687 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101dfba-2796-4428-b0dc-77b2709032ca None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:44:51.510889 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:51.563284 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b4906-9d75-41b9-9652-653548ebfbb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:44:51.563839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b4906-9d75-41b9-9652-653548ebfbb5 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:51.564117 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b4906-9d75-41b9-9652-653548ebfbb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:51.564385 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b4906-9d75-41b9-9652-653548ebfbb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:51.565961 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-ea0b4906-9d75-41b9-9652-653548ebfbb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:51.670339 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b4906-9d75-41b9-9652-653548ebfbb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:44:51.671754 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b4906-9d75-41b9-9652-653548ebfbb5 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:44:51.690714 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 566/1129] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:44:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:44:53.892723 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d00b83-4ed0-473e-9f04-e06089f91c14 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:53.893286 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d00b83-4ed0-473e-9f04-e06089f91c14 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:53.893286 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d00b83-4ed0-473e-9f04-e06089f91c14 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:53.894654 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 564/1130] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:44:53.900030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24263cd-a5b2-41d9-9e19-aed56061b268 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:53.900456 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24263cd-a5b2-41d9-9e19-aed56061b268 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:53.900616 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24263cd-a5b2-41d9-9e19-aed56061b268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:53.964823 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:53.967532 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-c24263cd-a5b2-41d9-9e19-aed56061b268 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:44:53.968817 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c24263cd-a5b2-41d9-9e19-aed56061b268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:54.063252 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 567/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:44:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:44:54.346168 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d00b83-4ed0-473e-9f04-e06089f91c14 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:44:54.348269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-b4d00b83-4ed0-473e-9f04-e06089f91c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:54.449610 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d00b83-4ed0-473e-9f04-e06089f91c14 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:44:54.498571 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:54.523577 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2819c940-e388-48dc-b146-ef88db7b9126 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:44:54.524211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819c940-e388-48dc-b146-ef88db7b9126 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:54.524489 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819c940-e388-48dc-b146-ef88db7b9126 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:54.524758 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819c940-e388-48dc-b146-ef88db7b9126 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:54.526351 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-2819c940-e388-48dc-b146-ef88db7b9126 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:54.634003 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2819c940-e388-48dc-b146-ef88db7b9126 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:44:54.635342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2819c940-e388-48dc-b146-ef88db7b9126 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:44:54.653979 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 565/1132] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:44:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:44:56.837087 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0131162-520e-485a-a76b-ce2db328e4d7 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:56.837667 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0131162-520e-485a-a76b-ce2db328e4d7 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:56.837667 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0131162-520e-485a-a76b-ce2db328e4d7 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:56.838979 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 568/1133] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:44:56.844544 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ded82-3cbd-4cf0-9099-fcdecc776ec9 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:56.844961 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ded82-3cbd-4cf0-9099-fcdecc776ec9 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:56.845108 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ded82-3cbd-4cf0-9099-fcdecc776ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:56.910355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-9d7ded82-3cbd-4cf0-9099-fcdecc776ec9 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:44:56.911558 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-9d7ded82-3cbd-4cf0-9099-fcdecc776ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:56.940707 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:57.008099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 566/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:44:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:44:57.291584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0131162-520e-485a-a76b-ce2db328e4d7 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:44:57.293139 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-f0131162-520e-485a-a76b-ce2db328e4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:57.394992 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0131162-520e-485a-a76b-ce2db328e4d7 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:44:57.422840 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:57.467058 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be4162c-23ad-46cb-ae61-0593c03238d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:44:57.467657 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4162c-23ad-46cb-ae61-0593c03238d3 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:57.467960 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4162c-23ad-46cb-ae61-0593c03238d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:57.468226 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be4162c-23ad-46cb-ae61-0593c03238d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:57.469796 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-9be4162c-23ad-46cb-ae61-0593c03238d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:57.575009 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be4162c-23ad-46cb-ae61-0593c03238d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:44:57.576393 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be4162c-23ad-46cb-ae61-0593c03238d3 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:44:57.594568 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 569/1135] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:44:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:44:59.778065 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70d311-fbf3-4c29-9d4f-9b0f3e4432bd None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:59.778594 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70d311-fbf3-4c29-9d4f-9b0f3e4432bd None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:59.778713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70d311-fbf3-4c29-9d4f-9b0f3e4432bd None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:59.780049 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 567/1136] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:44:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:44:59.785224 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff31a3-4800-452e-9a46-4e98792cc28a None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:44:59.785632 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff31a3-4800-452e-9a46-4e98792cc28a None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:44:59.785779 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff31a3-4800-452e-9a46-4e98792cc28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:44:59.852527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-abff31a3-4800-452e-9a46-4e98792cc28a None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:44:59.853735 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-abff31a3-4800-452e-9a46-4e98792cc28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:44:59.861958 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:44:59.949281 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 570/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:44:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:45:00.238311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70d311-fbf3-4c29-9d4f-9b0f3e4432bd None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:45:00.239864 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-1b70d311-fbf3-4c29-9d4f-9b0f3e4432bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:00.341238 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70d311-fbf3-4c29-9d4f-9b0f3e4432bd None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:45:00.395500 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:45:00.414668 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902dad52-664f-40d7-adc7-ab498072826c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:45:00.415268 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902dad52-664f-40d7-adc7-ab498072826c service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:00.415542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902dad52-664f-40d7-adc7-ab498072826c service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:00.415837 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902dad52-664f-40d7-adc7-ab498072826c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:00.417431 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-902dad52-664f-40d7-adc7-ab498072826c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:00.525489 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902dad52-664f-40d7-adc7-ab498072826c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:45:00.526841 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902dad52-664f-40d7-adc7-ab498072826c service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:45:00.545241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 568/1138] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:45:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:45:02.723700 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb70cd8-ef9d-4d92-86e3-bfd01902cfc1 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:02.724279 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb70cd8-ef9d-4d92-86e3-bfd01902cfc1 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:02.724397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb70cd8-ef9d-4d92-86e3-bfd01902cfc1 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:02.725714 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 571/1139] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:45:02.730934 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec32ce-5ec6-48b4-b111-7baf2f523cd6 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:02.731384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec32ce-5ec6-48b4-b111-7baf2f523cd6 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:02.731492 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec32ce-5ec6-48b4-b111-7baf2f523cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:02.796480 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-51ec32ce-5ec6-48b4-b111-7baf2f523cd6 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:45:02.797681 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-51ec32ce-5ec6-48b4-b111-7baf2f523cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:02.835762 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:45:02.893828 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 569/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:45:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:45:03.179936 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb70cd8-ef9d-4d92-86e3-bfd01902cfc1 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:45:03.181504 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-8eb70cd8-ef9d-4d92-86e3-bfd01902cfc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:03.282891 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb70cd8-ef9d-4d92-86e3-bfd01902cfc1 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:45:03.320001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:45:03.355076 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42caea6-02dc-49d1-99e8-843449b9f2ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:45:03.355677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42caea6-02dc-49d1-99e8-843449b9f2ee service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:03.356106 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42caea6-02dc-49d1-99e8-843449b9f2ee service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:03.356377 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42caea6-02dc-49d1-99e8-843449b9f2ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:03.357945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-f42caea6-02dc-49d1-99e8-843449b9f2ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:03.461743 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42caea6-02dc-49d1-99e8-843449b9f2ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:45:03.463085 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42caea6-02dc-49d1-99e8-843449b9f2ee service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:45:03.482120 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 572/1141] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:45:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:45:05.638940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c75fb1-de57-4ed4-badb-b3e3ffaa48d5 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:05.639466 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c75fb1-de57-4ed4-badb-b3e3ffaa48d5 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:05.639466 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c75fb1-de57-4ed4-badb-b3e3ffaa48d5 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:05.640856 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 570/1142] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:45:05.646885 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c05972-5352-4d74-8af2-9acbaecd9ca4 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:05.647287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c05972-5352-4d74-8af2-9acbaecd9ca4 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:05.647426 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c05972-5352-4d74-8af2-9acbaecd9ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:05.712938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-43c05972-5352-4d74-8af2-9acbaecd9ca4 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:45:05.714145 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-43c05972-5352-4d74-8af2-9acbaecd9ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:05.732048 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:45:05.810327 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 573/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:45:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:45:06.091444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c75fb1-de57-4ed4-badb-b3e3ffaa48d5 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:45:06.093066 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-a7c75fb1-de57-4ed4-badb-b3e3ffaa48d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:06.195744 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c75fb1-de57-4ed4-badb-b3e3ffaa48d5 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:45:06.261704 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:45:06.268177 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aaeed2-5154-4dd8-bd28-0491e0d4d27a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:45:06.271316 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaeed2-5154-4dd8-bd28-0491e0d4d27a service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:06.271316 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaeed2-5154-4dd8-bd28-0491e0d4d27a service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:06.271316 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaeed2-5154-4dd8-bd28-0491e0d4d27a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:06.271685 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-99aaeed2-5154-4dd8-bd28-0491e0d4d27a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:06.379718 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aaeed2-5154-4dd8-bd28-0491e0d4d27a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:45:06.381099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aaeed2-5154-4dd8-bd28-0491e0d4d27a service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:45:06.399363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 571/1144] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:45:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:45:08.541372 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079bff7-c723-4ef3-9fc3-410faa894526 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:08.541951 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079bff7-c723-4ef3-9fc3-410faa894526 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:08.541951 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b079bff7-c723-4ef3-9fc3-410faa894526 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:08.543249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 574/1145] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:45:08.548829 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa08a3f-5b85-4ecc-a7e5-5fee96fae873 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:08.549263 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa08a3f-5b85-4ecc-a7e5-5fee96fae873 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:08.549445 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa08a3f-5b85-4ecc-a7e5-5fee96fae873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:08.615501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-1fa08a3f-5b85-4ecc-a7e5-5fee96fae873 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:45:08.616752 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-1fa08a3f-5b85-4ecc-a7e5-5fee96fae873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:08.663503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:45:08.711330 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 572/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:45:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:45:08.993621 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079bff7-c723-4ef3-9fc3-410faa894526 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:45:08.995163 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-b079bff7-c723-4ef3-9fc3-410faa894526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:09.097603 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b079bff7-c723-4ef3-9fc3-410faa894526 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:45:09.144490 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:45:09.169885 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66eca553-9e14-4aef-91ae-97d648c37eda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:45:09.170496 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eca553-9e14-4aef-91ae-97d648c37eda service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:09.170773 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eca553-9e14-4aef-91ae-97d648c37eda service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:09.171044 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66eca553-9e14-4aef-91ae-97d648c37eda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:09.172706 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-66eca553-9e14-4aef-91ae-97d648c37eda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:09.278521 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eca553-9e14-4aef-91ae-97d648c37eda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:45:09.279983 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66eca553-9e14-4aef-91ae-97d648c37eda service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:45:09.298106 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 575/1147] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:45:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:45:18.709821 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=22 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:45:19.295822 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=66 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:45:29.327124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245a9ba-bbf7-46dd-bf61-4cf8fb1a1843 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:29.327610 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245a9ba-bbf7-46dd-bf61-4cf8fb1a1843 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:29.327715 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245a9ba-bbf7-46dd-bf61-4cf8fb1a1843 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:29.329052 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 573/1148] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:45:29.334351 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109f773-97c0-4a05-ac38-e74b6bdd29f9 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:29.334762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109f773-97c0-4a05-ac38-e74b6bdd29f9 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:29.334901 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7109f773-97c0-4a05-ac38-e74b6bdd29f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:29.399426 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-7109f773-97c0-4a05-ac38-e74b6bdd29f9 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:45:29.400690 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-7109f773-97c0-4a05-ac38-e74b6bdd29f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:29.496768 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 576/1149] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:45:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:45:29.794496 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281c322-c760-455f-bf66-bb9ac426eae7 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:29.794784 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281c322-c760-455f-bf66-bb9ac426eae7 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:29.795061 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281c322-c760-455f-bf66-bb9ac426eae7 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:29.796453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 574/1150] 10.222.0.17 () {54 vars in 962 bytes} [Mon Oct 17 03:45:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:45:29.801961 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515befe-32bb-42a4-ad60-dd94081eae29 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:29.802244 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515befe-32bb-42a4-ad60-dd94081eae29 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:29.802517 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515befe-32bb-42a4-ad60-dd94081eae29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:29.875750 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-0515befe-32bb-42a4-ad60-dd94081eae29 None None] MFA Rules not processed for user `565a836c7f89478cad615bd6c14531a7`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:45:29.876954 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-0515befe-32bb-42a4-ad60-dd94081eae29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:29.896977 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:45:29.950996 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 577/1151] 10.222.0.17 () {58 vars in 1059 bytes} [Mon Oct 17 03:45:29 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:45:29.959333 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281c322-c760-455f-bf66-bb9ac426eae7 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:45:29.961779 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6281c322-c760-455f-bf66-bb9ac426eae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:30.064074 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281c322-c760-455f-bf66-bb9ac426eae7 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:45:30.138360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b909af-4efb-493f-bf51-77d33ecb0936 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '565a836c7f89478cad615bd6c14531a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:45:30.138962 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b909af-4efb-493f-bf51-77d33ecb0936 service glance] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:30.139243 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b909af-4efb-493f-bf51-77d33ecb0936 service glance] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:30.139512 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b909af-4efb-493f-bf51-77d33ecb0936 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:30.141137 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-41b909af-4efb-493f-bf51-77d33ecb0936 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:30.197122 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:45:30.252535 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b909af-4efb-493f-bf51-77d33ecb0936 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:45:30.254560 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b909af-4efb-493f-bf51-77d33ecb0936 service glance] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:45:30.273785 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 575/1152] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:45:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:45:32.488053 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a04b4-7876-42d2-bf92-f4a48ef79008 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:32.495039 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a04b4-7876-42d2-bf92-f4a48ef79008 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:32.495039 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a04b4-7876-42d2-bf92-f4a48ef79008 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:32.495039 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 578/1153] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:45:32.495327 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29669458-8869-4185-8713-48250e72c571 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:32.495712 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29669458-8869-4185-8713-48250e72c571 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:32.495712 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29669458-8869-4185-8713-48250e72c571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:32.560481 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-29669458-8869-4185-8713-48250e72c571 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:45:32.561682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-29669458-8869-4185-8713-48250e72c571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:32.657070 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 576/1154] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:45:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:45:32.938214 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a04b4-7876-42d2-bf92-f4a48ef79008 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:45:32.938214 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-514a04b4-7876-42d2-bf92-f4a48ef79008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:33.057295 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a04b4-7876-42d2-bf92-f4a48ef79008 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:45:33.105830 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:45:33.148642 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375da78-9438-4e3a-a30a-b956ff82769d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:45:33.149246 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375da78-9438-4e3a-a30a-b956ff82769d admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:33.149529 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375da78-9438-4e3a-a30a-b956ff82769d admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:33.149796 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375da78-9438-4e3a-a30a-b956ff82769d admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:33.150419 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375da78-9438-4e3a-a30a-b956ff82769d admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:45:33.151495 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: WARNING py.warnings [None req-8375da78-9438-4e3a-a30a-b956ff82769d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 03:45:33.151495 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: warnings.warn(msg) Oct 17 03:45:33.151495 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: Oct 17 03:45:33.152648 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375da78-9438-4e3a-a30a-b956ff82769d admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:45:33.171519 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 579/1155] 10.222.0.17 () {56 vars in 1108 bytes} [Mon Oct 17 03:45:32 2022] GET /identity/v3/projects => generated 2553 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:45:35.300384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034fe7c-0f26-47dc-98a8-f2ec43756431 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:35.300840 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034fe7c-0f26-47dc-98a8-f2ec43756431 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:35.300942 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034fe7c-0f26-47dc-98a8-f2ec43756431 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:35.302292 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 577/1156] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:45:35.307588 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0f911-773b-4ac8-bbeb-d62824474628 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:35.308007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0f911-773b-4ac8-bbeb-d62824474628 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:35.308188 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b0f911-773b-4ac8-bbeb-d62824474628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:35.375129 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-38b0f911-773b-4ac8-bbeb-d62824474628 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:45:35.376952 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-38b0f911-773b-4ac8-bbeb-d62824474628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:35.470860 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 580/1157] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:45:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:45:35.765820 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034fe7c-0f26-47dc-98a8-f2ec43756431 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:45:35.767406 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-3034fe7c-0f26-47dc-98a8-f2ec43756431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:35.777208 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:45:35.912605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034fe7c-0f26-47dc-98a8-f2ec43756431 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:45:35.991059 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:45:36.023287 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a90d0-bcd2-4caa-9589-342003a27367 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:45:36.023917 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a90d0-bcd2-4caa-9589-342003a27367 service nova] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:36.024196 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a90d0-bcd2-4caa-9589-342003a27367 service nova] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:36.024467 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a90d0-bcd2-4caa-9589-342003a27367 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:36.026074 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-bb7a90d0-bcd2-4caa-9589-342003a27367 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:36.114542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a90d0-bcd2-4caa-9589-342003a27367 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:45:36.116168 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a90d0-bcd2-4caa-9589-342003a27367 service nova] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:45:36.133515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 578/1158] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:45:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:45:38.423533 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941644c-a387-48d2-a428-8dcf08dc03f1 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:38.424111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941644c-a387-48d2-a428-8dcf08dc03f1 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:38.424111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d941644c-a387-48d2-a428-8dcf08dc03f1 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:38.425440 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 581/1159] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:45:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:45:38.430818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40dd54c-67a0-4cec-b871-f5584e028701 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:38.431214 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40dd54c-67a0-4cec-b871-f5584e028701 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:38.431372 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40dd54c-67a0-4cec-b871-f5584e028701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:38.496657 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-d40dd54c-67a0-4cec-b871-f5584e028701 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:45:38.497865 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-d40dd54c-67a0-4cec-b871-f5584e028701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:38.543173 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:45:38.592725 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 579/1160] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:45:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:45:38.882899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941644c-a387-48d2-a428-8dcf08dc03f1 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:45:38.884546 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-d941644c-a387-48d2-a428-8dcf08dc03f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:38.908944 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:45:38.987232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d941644c-a387-48d2-a428-8dcf08dc03f1 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:45:39.058695 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e22db-2d48-4269-beb7-c25067bb4a25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:45:39.059292 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e22db-2d48-4269-beb7-c25067bb4a25 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:39.059568 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e22db-2d48-4269-beb7-c25067bb4a25 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:39.059898 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e22db-2d48-4269-beb7-c25067bb4a25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:39.061466 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-4b5e22db-2d48-4269-beb7-c25067bb4a25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:39.133925 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:45:39.167727 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e22db-2d48-4269-beb7-c25067bb4a25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:45:39.169163 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e22db-2d48-4269-beb7-c25067bb4a25 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:45:39.187025 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 582/1161] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:45:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:45:41.379523 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebc7ea-f70d-48c0-b8bd-2f28ff4d6e21 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:41.380034 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebc7ea-f70d-48c0-b8bd-2f28ff4d6e21 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:41.380151 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ebc7ea-f70d-48c0-b8bd-2f28ff4d6e21 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:41.381494 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 580/1162] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:45:41.386699 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb06b92-7a3d-4a7c-a280-2b8e6138a110 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:41.387100 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb06b92-7a3d-4a7c-a280-2b8e6138a110 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:41.387259 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb06b92-7a3d-4a7c-a280-2b8e6138a110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:41.454288 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-3cb06b92-7a3d-4a7c-a280-2b8e6138a110 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:45:41.455490 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-3cb06b92-7a3d-4a7c-a280-2b8e6138a110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:41.550811 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 583/1163] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:45:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:45:41.842625 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ebc7ea-f70d-48c0-b8bd-2f28ff4d6e21 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:45:41.844199 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-29ebc7ea-f70d-48c0-b8bd-2f28ff4d6e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:41.947007 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ebc7ea-f70d-48c0-b8bd-2f28ff4d6e21 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:45:42.019588 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:45:42.021082 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f74d11-d949-4fdb-bd6b-c935a1e8c1b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:45:42.021691 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f74d11-d949-4fdb-bd6b-c935a1e8c1b6 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:42.021969 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f74d11-d949-4fdb-bd6b-c935a1e8c1b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:42.022239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f74d11-d949-4fdb-bd6b-c935a1e8c1b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:42.023884 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-06f74d11-d949-4fdb-bd6b-c935a1e8c1b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:42.132894 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f74d11-d949-4fdb-bd6b-c935a1e8c1b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:45:42.134245 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f74d11-d949-4fdb-bd6b-c935a1e8c1b6 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:45:42.152740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 581/1164] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:45:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:45:44.372477 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfa71f-3210-4809-a9fd-84f9c013bd01 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:44.373035 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfa71f-3210-4809-a9fd-84f9c013bd01 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:44.373035 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfa71f-3210-4809-a9fd-84f9c013bd01 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:44.374359 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 584/1165] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:45:44.379572 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6317f9-3439-4fa5-aef9-8aab61001e97 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:44.380049 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6317f9-3439-4fa5-aef9-8aab61001e97 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:44.380147 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6317f9-3439-4fa5-aef9-8aab61001e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:44.445976 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-3c6317f9-3439-4fa5-aef9-8aab61001e97 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:45:44.447204 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-3c6317f9-3439-4fa5-aef9-8aab61001e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:44.499339 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:45:44.543017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 582/1166] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:45:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:45:44.830949 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bfa71f-3210-4809-a9fd-84f9c013bd01 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:45:44.832522 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-a3bfa71f-3210-4809-a9fd-84f9c013bd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:44.868163 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:45:44.934956 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bfa71f-3210-4809-a9fd-84f9c013bd01 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:45:45.008260 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bae6fd-9cb9-45bb-a8e0-2820f8dd025d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:45:45.008859 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bae6fd-9cb9-45bb-a8e0-2820f8dd025d service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:45.009139 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bae6fd-9cb9-45bb-a8e0-2820f8dd025d service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:45.009407 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bae6fd-9cb9-45bb-a8e0-2820f8dd025d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:45.010996 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-47bae6fd-9cb9-45bb-a8e0-2820f8dd025d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:45.088604 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:45:45.117600 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bae6fd-9cb9-45bb-a8e0-2820f8dd025d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:45:45.119025 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bae6fd-9cb9-45bb-a8e0-2820f8dd025d service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:45:45.137271 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 585/1167] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:45:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:45:47.550661 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9c93f-cfc1-40a3-85ab-8b214e923f20 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:47.551200 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9c93f-cfc1-40a3-85ab-8b214e923f20 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:47.551200 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9c93f-cfc1-40a3-85ab-8b214e923f20 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:47.552500 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 583/1168] 10.222.0.17 () {54 vars in 927 bytes} [Mon Oct 17 03:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:45:47.558054 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df72f5a-2a2d-459e-8823-ffee57419cf7 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:47.558479 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df72f5a-2a2d-459e-8823-ffee57419cf7 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:47.558616 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df72f5a-2a2d-459e-8823-ffee57419cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:47.623701 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-4df72f5a-2a2d-459e-8823-ffee57419cf7 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:45:47.624941 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-4df72f5a-2a2d-459e-8823-ffee57419cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:47.722297 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 586/1169] 10.222.0.17 () {58 vars in 1024 bytes} [Mon Oct 17 03:45:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:45:48.017374 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9c93f-cfc1-40a3-85ab-8b214e923f20 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:45:48.018904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-b1e9c93f-cfc1-40a3-85ab-8b214e923f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:48.120884 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9c93f-cfc1-40a3-85ab-8b214e923f20 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:45:48.194619 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146f948-28d7-43e0-941b-9f183df80c88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:45:48.195220 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146f948-28d7-43e0-941b-9f183df80c88 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:45:48.195501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146f948-28d7-43e0-941b-9f183df80c88 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:45:48.195801 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146f948-28d7-43e0-941b-9f183df80c88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:45:48.197406 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-c146f948-28d7-43e0-941b-9f183df80c88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:45:48.205837 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:45:48.306294 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c146f948-28d7-43e0-941b-9f183df80c88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:45:48.307655 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c146f948-28d7-43e0-941b-9f183df80c88 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:45:48.325212 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 584/1170] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:45:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:45:57.720789 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=82 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:45:58.322951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=50 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:47:16.757980 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e10d63-f519-441a-889b-6cfee2ea5c7b None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:47:16.758498 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e10d63-f519-441a-889b-6cfee2ea5c7b None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:47:16.758498 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e10d63-f519-441a-889b-6cfee2ea5c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:47:16.832886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-77e10d63-f519-441a-889b-6cfee2ea5c7b None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:47:16.834108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-77e10d63-f519-441a-889b-6cfee2ea5c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:47:16.928148 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 587/1171] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:47:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:47:16.936753 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ea90c-795b-429e-8d9e-98d895a3cac4 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:47:16.937234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ea90c-795b-429e-8d9e-98d895a3cac4 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:47:16.937234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ea90c-795b-429e-8d9e-98d895a3cac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:47:17.012098 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-ab0ea90c-795b-429e-8d9e-98d895a3cac4 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:47:17.013311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-ab0ea90c-795b-429e-8d9e-98d895a3cac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:47:17.109566 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 585/1172] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:47:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:47:17.114169 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e10d63-f519-441a-889b-6cfee2ea5c7b None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:47:17.115454 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-77e10d63-f519-441a-889b-6cfee2ea5c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:47:17.126845 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e10d63-f519-441a-889b-6cfee2ea5c7b None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:47:17.187559 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:47:17.221679 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914cd74-760a-463c-bbc9-4002cecde997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:47:17.222277 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914cd74-760a-463c-bbc9-4002cecde997 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:47:17.222552 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914cd74-760a-463c-bbc9-4002cecde997 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:47:17.222819 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914cd74-760a-463c-bbc9-4002cecde997 admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:47:17.223445 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b914cd74-760a-463c-bbc9-4002cecde997 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:47:17.223764 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b914cd74-760a-463c-bbc9-4002cecde997 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:47:17.224758 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: WARNING py.warnings [None req-b914cd74-760a-463c-bbc9-4002cecde997 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 17 03:47:17.224758 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: warnings.warn(msg) Oct 17 03:47:17.224758 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: Oct 17 03:47:17.225625 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b914cd74-760a-463c-bbc9-4002cecde997 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:47:17.237067 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 588/1173] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:47:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:47:17.246490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ea90c-795b-429e-8d9e-98d895a3cac4 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:47:17.255568 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ea90c-795b-429e-8d9e-98d895a3cac4 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:47:17.316962 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:47:17.349910 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb65b0-3830-476c-9b4d-cf26ef9e1fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:47:17.350507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb65b0-3830-476c-9b4d-cf26ef9e1fff admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:47:17.350788 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb65b0-3830-476c-9b4d-cf26ef9e1fff admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:47:17.351054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb65b0-3830-476c-9b4d-cf26ef9e1fff admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:47:17.351718 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fb65b0-3830-476c-9b4d-cf26ef9e1fff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:47:17.352786 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: WARNING py.warnings [None req-33fb65b0-3830-476c-9b4d-cf26ef9e1fff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 03:47:17.352786 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: warnings.warn(msg) Oct 17 03:47:17.352786 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: Oct 17 03:47:17.353977 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fb65b0-3830-476c-9b4d-cf26ef9e1fff admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:47:17.460997 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 586/1174] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:47:17 2022] POST /identity/v3/projects => generated 377 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:47:17.465943 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914cd74-760a-463c-bbc9-4002cecde997 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:47:17.467460 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-b914cd74-760a-463c-bbc9-4002cecde997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:47:17.500954 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914cd74-760a-463c-bbc9-4002cecde997 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:47:17.594447 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19081bc-92bf-4600-96f8-e25df40d562a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:47:17.595049 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19081bc-92bf-4600-96f8-e25df40d562a admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:47:17.595328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19081bc-92bf-4600-96f8-e25df40d562a admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:47:17.595598 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19081bc-92bf-4600-96f8-e25df40d562a admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:47:17.596259 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19081bc-92bf-4600-96f8-e25df40d562a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:47:17.597257 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: WARNING py.warnings [None req-c19081bc-92bf-4600-96f8-e25df40d562a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 03:47:17.597257 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: warnings.warn(msg) Oct 17 03:47:17.597257 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: Oct 17 03:47:17.598481 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19081bc-92bf-4600-96f8-e25df40d562a admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:47:17.695897 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 589/1175] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:47:17 2022] POST /identity/v3/users => generated 289 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:47:17.700689 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb65b0-3830-476c-9b4d-cf26ef9e1fff admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:47:17.702217 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-33fb65b0-3830-476c-9b4d-cf26ef9e1fff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:47:17.763624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb65b0-3830-476c-9b4d-cf26ef9e1fff admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:47:17.846504 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:47:17.856446 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68c6c4-117c-4fd9-b9bf-f5805ed3b0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:47:17.860753 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68c6c4-117c-4fd9-b9bf-f5805ed3b0ab admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:47:17.860753 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68c6c4-117c-4fd9-b9bf-f5805ed3b0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:47:17.860753 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68c6c4-117c-4fd9-b9bf-f5805ed3b0ab admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:47:17.860753 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68c6c4-117c-4fd9-b9bf-f5805ed3b0ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:47:17.860753 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: WARNING py.warnings [None req-ee68c6c4-117c-4fd9-b9bf-f5805ed3b0ab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 17 03:47:17.860753 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: warnings.warn(msg) Oct 17 03:47:17.860753 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: Oct 17 03:47:17.861362 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee68c6c4-117c-4fd9-b9bf-f5805ed3b0ab admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:47:17.874235 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 587/1176] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:47:17 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:47:17.878138 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19081bc-92bf-4600-96f8-e25df40d562a admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:47:17.879641 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c19081bc-92bf-4600-96f8-e25df40d562a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:47:17.891587 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19081bc-92bf-4600-96f8-e25df40d562a admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:47:17.916152 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:47:17.986830 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac2cf1-d73d-4b0b-937f-501349904ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:47:17.987451 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac2cf1-d73d-4b0b-937f-501349904ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:47:17.987706 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac2cf1-d73d-4b0b-937f-501349904ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:47:17.988007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac2cf1-d73d-4b0b-937f-501349904ec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:47:17.988639 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac2cf1-d73d-4b0b-937f-501349904ec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:47:17.990211 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac2cf1-d73d-4b0b-937f-501349904ec2 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:47:18.002587 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 590/1177] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:47:17 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:47:18.006753 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68c6c4-117c-4fd9-b9bf-f5805ed3b0ab admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:47:18.008317 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-ee68c6c4-117c-4fd9-b9bf-f5805ed3b0ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:47:18.041735 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68c6c4-117c-4fd9-b9bf-f5805ed3b0ab admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:47:18.135190 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0131edd-58fc-4984-815b-08f410078124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:47:18.136475 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0131edd-58fc-4984-815b-08f410078124 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:47:18.136758 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0131edd-58fc-4984-815b-08f410078124 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:47:18.137030 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0131edd-58fc-4984-815b-08f410078124 admin admin] PATH_INFO: `/v3/projects/8a401eefb0e4445fa658753b2a47a6a1/users/3b27fac3ce3340bf9ef1b2941cb1542c/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:47:18.159019 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0131edd-58fc-4984-815b-08f410078124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a401eefb0e4445fa658753b2a47a6a1, user_id=3b27fac3ce3340bf9ef1b2941cb1542c, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:47:18.161689 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0131edd-58fc-4984-815b-08f410078124 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:47:18.265227 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 588/1178] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:47:18 2022] PUT /identity/v3/projects/8a401eefb0e4445fa658753b2a47a6a1/users/3b27fac3ce3340bf9ef1b2941cb1542c/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:47:18.276326 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aed032-6340-4d99-a7a1-28417c75621e None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:47:18.276615 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aed032-6340-4d99-a7a1-28417c75621e None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:47:18.276916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aed032-6340-4d99-a7a1-28417c75621e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:47:18.332674 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-50aed032-6340-4d99-a7a1-28417c75621e None None] MFA Rules not processed for user `3b27fac3ce3340bf9ef1b2941cb1542c`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:47:18.333801 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-50aed032-6340-4d99-a7a1-28417c75621e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:47:18.408616 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 591/1179] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:47:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:47:18.421957 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51d5d0-1b5d-47f4-83ae-89dd91c3b954 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:47:18.422239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51d5d0-1b5d-47f4-83ae-89dd91c3b954 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:47:18.422507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51d5d0-1b5d-47f4-83ae-89dd91c3b954 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:47:18.423875 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 589/1180] 10.222.0.17 () {54 vars in 963 bytes} [Mon Oct 17 03:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:47:18.428442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e074d34-7faf-49a0-a05a-5b2f76ef9aba None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:47:18.428726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e074d34-7faf-49a0-a05a-5b2f76ef9aba None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:47:18.428997 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e074d34-7faf-49a0-a05a-5b2f76ef9aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:47:18.459545 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:47:18.503683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-6e074d34-7faf-49a0-a05a-5b2f76ef9aba None None] MFA Rules not processed for user `99bc94105bbf4cb4a60e37133edc8dfb`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:47:18.504907 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-6e074d34-7faf-49a0-a05a-5b2f76ef9aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:47:18.579151 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 592/1181] 10.222.0.17 () {58 vars in 1060 bytes} [Mon Oct 17 03:47:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:47:18.586850 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51d5d0-1b5d-47f4-83ae-89dd91c3b954 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:47:18.588699 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-3d51d5d0-1b5d-47f4-83ae-89dd91c3b954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:47:18.638552 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:47:18.692264 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51d5d0-1b5d-47f4-83ae-89dd91c3b954 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:47:18.765129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe49e4-4421-41ec-ad23-1afc7b0ee59b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:47:18.765733 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe49e4-4421-41ec-ad23-1afc7b0ee59b service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:47:18.766014 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe49e4-4421-41ec-ad23-1afc7b0ee59b service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:47:18.766289 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe49e4-4421-41ec-ad23-1afc7b0ee59b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:47:18.767916 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-e5fe49e4-4421-41ec-ad23-1afc7b0ee59b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:47:18.866153 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe49e4-4421-41ec-ad23-1afc7b0ee59b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:47:18.867514 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe49e4-4421-41ec-ad23-1afc7b0ee59b service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:47:18.871018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:47:18.885096 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 590/1182] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:47:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:47:18.930695 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00eb1b-5257-461c-a9c5-393b13d73cf7 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:47:18.931097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00eb1b-5257-461c-a9c5-393b13d73cf7 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:47:18.931271 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00eb1b-5257-461c-a9c5-393b13d73cf7 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:47:18.932570 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 593/1183] 10.222.0.17 () {52 vars in 830 bytes} [Mon Oct 17 03:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:47:18.936940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe49e4-4421-41ec-ad23-1afc7b0ee59b service cinder] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:47:18.938206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-e5fe49e4-4421-41ec-ad23-1afc7b0ee59b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:47:19.035787 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe49e4-4421-41ec-ad23-1afc7b0ee59b service cinder] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:47:19.127036 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9199d1-9e59-42a2-99a9-9a01c7c4fb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:47:19.127581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9199d1-9e59-42a2-99a9-9a01c7c4fb58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:47:19.127889 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9199d1-9e59-42a2-99a9-9a01c7c4fb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:47:19.128161 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9199d1-9e59-42a2-99a9-9a01c7c4fb58 admin admin] PATH_INFO: `/v3/users/3b27fac3ce3340bf9ef1b2941cb1542c` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:47:19.141099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9199d1-9e59-42a2-99a9-9a01c7c4fb58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b27fac3ce3340bf9ef1b2941cb1542c)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:47:19.142107 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: WARNING py.warnings [None req-9b9199d1-9e59-42a2-99a9-9a01c7c4fb58 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 03:47:19.142107 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: warnings.warn(msg) Oct 17 03:47:19.142107 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: Oct 17 03:47:19.143320 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9199d1-9e59-42a2-99a9-9a01c7c4fb58 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:47:19.153560 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:47:19.309281 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-9b9199d1-9e59-42a2-99a9-9a01c7c4fb58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b27fac3ce3340bf9ef1b2941cb1542c'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:47:19.309734 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-9b9199d1-9e59-42a2-99a9-9a01c7c4fb58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b27fac3ce3340bf9ef1b2941cb1542c'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:47:19.356297 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-9b9199d1-9e59-42a2-99a9-9a01c7c4fb58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b27fac3ce3340bf9ef1b2941cb1542c'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:47:19.380765 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-9b9199d1-9e59-42a2-99a9-9a01c7c4fb58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b27fac3ce3340bf9ef1b2941cb1542c'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:47:19.381265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-9b9199d1-9e59-42a2-99a9-9a01c7c4fb58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b27fac3ce3340bf9ef1b2941cb1542c'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:47:19.401379 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 591/1184] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:47:18 2022] DELETE /identity/v3/users/3b27fac3ce3340bf9ef1b2941cb1542c => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:47:19.405696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d05d2-75be-45e6-9801-7be7b5179601 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:47:19.405984 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d05d2-75be-45e6-9801-7be7b5179601 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:47:19.406262 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858d05d2-75be-45e6-9801-7be7b5179601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:47:19.462651 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-858d05d2-75be-45e6-9801-7be7b5179601 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:47:19.463909 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-858d05d2-75be-45e6-9801-7be7b5179601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:47:19.538988 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:47:19.554694 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 594/1185] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:47:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:47:19.565883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9199d1-9e59-42a2-99a9-9a01c7c4fb58 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:47:19.567145 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-9b9199d1-9e59-42a2-99a9-9a01c7c4fb58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:47:19.667798 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9199d1-9e59-42a2-99a9-9a01c7c4fb58 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:47:19.722164 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:47:19.741082 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a6b90-bf75-49e5-8431-4e10c4a6686a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:47:19.741697 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a6b90-bf75-49e5-8431-4e10c4a6686a service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:47:19.741972 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a6b90-bf75-49e5-8431-4e10c4a6686a service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:47:19.742242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a6b90-bf75-49e5-8431-4e10c4a6686a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:47:19.743862 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-207a6b90-bf75-49e5-8431-4e10c4a6686a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:47:19.849689 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207a6b90-bf75-49e5-8431-4e10c4a6686a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:47:19.851110 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207a6b90-bf75-49e5-8431-4e10c4a6686a service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:47:19.869407 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 592/1186] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:47:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:47:20.358977 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d05d2-75be-45e6-9801-7be7b5179601 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:47:20.360250 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-858d05d2-75be-45e6-9801-7be7b5179601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:47:20.371958 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858d05d2-75be-45e6-9801-7be7b5179601 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:47:20.468869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075d56f-5dfc-4a6b-acb3-2373a5b307d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:47:20.469502 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075d56f-5dfc-4a6b-acb3-2373a5b307d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:47:20.469779 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075d56f-5dfc-4a6b-acb3-2373a5b307d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:47:20.470051 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075d56f-5dfc-4a6b-acb3-2373a5b307d2 admin admin] PATH_INFO: `/v3/projects/8a401eefb0e4445fa658753b2a47a6a1` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:47:20.479767 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075d56f-5dfc-4a6b-acb3-2373a5b307d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a401eefb0e4445fa658753b2a47a6a1)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:47:20.480818 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: WARNING py.warnings [None req-f075d56f-5dfc-4a6b-acb3-2373a5b307d2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 03:47:20.480818 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: warnings.warn(msg) Oct 17 03:47:20.480818 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: Oct 17 03:47:20.482067 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075d56f-5dfc-4a6b-acb3-2373a5b307d2 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:47:20.584004 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f075d56f-5dfc-4a6b-acb3-2373a5b307d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a401eefb0e4445fa658753b2a47a6a1', 'request_id': 'req-f075d56f-5dfc-4a6b-acb3-2373a5b307d2'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:47:20.636936 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f075d56f-5dfc-4a6b-acb3-2373a5b307d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a401eefb0e4445fa658753b2a47a6a1', 'request_id': 'req-f075d56f-5dfc-4a6b-acb3-2373a5b307d2'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:47:20.643202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f075d56f-5dfc-4a6b-acb3-2373a5b307d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a401eefb0e4445fa658753b2a47a6a1', 'request_id': 'req-f075d56f-5dfc-4a6b-acb3-2373a5b307d2'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:47:20.673606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f075d56f-5dfc-4a6b-acb3-2373a5b307d2 admin admin] The token cache is being invalidate because project 8a401eefb0e4445fa658753b2a47a6a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:47:20.673923 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f075d56f-5dfc-4a6b-acb3-2373a5b307d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:47:20.674451 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f075d56f-5dfc-4a6b-acb3-2373a5b307d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:47:20.675781 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 595/1187] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:47:20 2022] DELETE /identity/v3/projects/8a401eefb0e4445fa658753b2a47a6a1 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:47:29.867193 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=62 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:47:30.671674 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:48:44.289236 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c539b2-507a-4f09-920f-da270deecbda None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:48:44.290179 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c539b2-507a-4f09-920f-da270deecbda None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:48:44.290410 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c539b2-507a-4f09-920f-da270deecbda None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:48:44.291845 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 593/1188] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 03:48:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:48:44.296794 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef6a8b-08cf-4f88-88da-0c0749fe79f4 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:48:44.297204 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef6a8b-08cf-4f88-88da-0c0749fe79f4 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:48:44.297350 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef6a8b-08cf-4f88-88da-0c0749fe79f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:48:44.377866 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-52ef6a8b-08cf-4f88-88da-0c0749fe79f4 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:48:44.379071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-52ef6a8b-08cf-4f88-88da-0c0749fe79f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:48:44.481204 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 596/1189] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 03:48:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:48:44.632694 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c539b2-507a-4f09-920f-da270deecbda None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:48:44.636730 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-d8c539b2-507a-4f09-920f-da270deecbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:48:44.753194 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c539b2-507a-4f09-920f-da270deecbda None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:48:44.758522 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0322918-1946-43ef-8150-2c009b2aea2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:48:44.759121 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0322918-1946-43ef-8150-2c009b2aea2f service nova] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:48:44.759399 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0322918-1946-43ef-8150-2c009b2aea2f service nova] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:48:44.759672 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0322918-1946-43ef-8150-2c009b2aea2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:48:44.767598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0322918-1946-43ef-8150-2c009b2aea2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:48:44.769219 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0322918-1946-43ef-8150-2c009b2aea2f service nova] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:48:44.778835 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 594/1190] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:48:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:48:45.853474 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c2b4b-d4e2-4f72-9249-81909997a0e2 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:48:45.854042 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c2b4b-d4e2-4f72-9249-81909997a0e2 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:48:45.854042 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c2b4b-d4e2-4f72-9249-81909997a0e2 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:48:45.855361 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 597/1191] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 03:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:48:45.861573 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c79b83-07a7-4211-a6eb-3f270f7ca236 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:48:45.862005 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c79b83-07a7-4211-a6eb-3f270f7ca236 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:48:45.862129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c79b83-07a7-4211-a6eb-3f270f7ca236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:48:45.881633 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-87c79b83-07a7-4211-a6eb-3f270f7ca236 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:48:45.882881 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-87c79b83-07a7-4211-a6eb-3f270f7ca236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:48:45.890725 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 595/1192] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 03:48:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:48:46.043179 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c2b4b-d4e2-4f72-9249-81909997a0e2 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:48:46.058219 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c2b4b-d4e2-4f72-9249-81909997a0e2 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:48:46.063765 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee5743-685d-45b4-a62a-ded3502d103f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:48:46.064389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee5743-685d-45b4-a62a-ded3502d103f service nova] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:48:46.064656 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee5743-685d-45b4-a62a-ded3502d103f service nova] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:48:46.064929 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee5743-685d-45b4-a62a-ded3502d103f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:48:46.072948 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ee5743-685d-45b4-a62a-ded3502d103f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:48:46.074520 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ee5743-685d-45b4-a62a-ded3502d103f service nova] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:48:46.083856 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 598/1193] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:48:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:48:46.598684 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b1972-8af5-4c81-a7f8-82ecf3e99182 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:48:46.598987 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b1972-8af5-4c81-a7f8-82ecf3e99182 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:48:46.599246 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88b1972-8af5-4c81-a7f8-82ecf3e99182 None None] PATH_INFO: `/v3` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:48:46.601268 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 596/1194] 10.222.0.33 () {54 vars in 925 bytes} [Mon Oct 17 03:48:46 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:48:46.608431 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a10002-03b4-41bb-bc59-e279491bb4bc None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:48:46.608710 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a10002-03b4-41bb-bc59-e279491bb4bc None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:48:46.608979 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a10002-03b4-41bb-bc59-e279491bb4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:48:46.667456 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-c0a10002-03b4-41bb-bc59-e279491bb4bc None None] MFA Rules not processed for user `b500a113a2aa4a95875d7e094b917db3`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:48:46.668703 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c0a10002-03b4-41bb-bc59-e279491bb4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:48:46.685724 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:48:46.739381 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 599/1195] 10.222.0.33 () {58 vars in 1015 bytes} [Mon Oct 17 03:48:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:48:50.657955 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490acbfc-08bd-4344-b047-60e35e5f1b54 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:48:50.658415 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490acbfc-08bd-4344-b047-60e35e5f1b54 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:48:50.658510 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490acbfc-08bd-4344-b047-60e35e5f1b54 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:48:50.660063 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 597/1196] 10.222.0.17 () {54 vars in 961 bytes} [Mon Oct 17 03:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:48:50.665092 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da235e-c1a7-4e39-b873-8808b67485f9 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:48:50.665553 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da235e-c1a7-4e39-b873-8808b67485f9 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:48:50.665703 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da235e-c1a7-4e39-b873-8808b67485f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:48:50.685214 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-04da235e-c1a7-4e39-b873-8808b67485f9 None None] MFA Rules not processed for user `b500a113a2aa4a95875d7e094b917db3`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:48:50.686406 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-04da235e-c1a7-4e39-b873-8808b67485f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:48:50.695314 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 600/1197] 10.222.0.17 () {58 vars in 1058 bytes} [Mon Oct 17 03:48:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:48:50.702821 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490acbfc-08bd-4344-b047-60e35e5f1b54 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:48:50.716612 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490acbfc-08bd-4344-b047-60e35e5f1b54 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:48:50.721470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1250a793-6320-419e-af32-dcabee0da56a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b500a113a2aa4a95875d7e094b917db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:48:50.722059 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250a793-6320-419e-af32-dcabee0da56a service placement] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:48:50.722339 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250a793-6320-419e-af32-dcabee0da56a service placement] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:48:50.722609 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250a793-6320-419e-af32-dcabee0da56a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:48:50.730652 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1250a793-6320-419e-af32-dcabee0da56a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:48:50.732247 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1250a793-6320-419e-af32-dcabee0da56a service placement] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:48:50.742044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 598/1198] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:48:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:48:57.557923 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190bd5e-d706-4dfb-a25f-8ecdd2b962c8 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:48:57.558556 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190bd5e-d706-4dfb-a25f-8ecdd2b962c8 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:48:57.558556 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190bd5e-d706-4dfb-a25f-8ecdd2b962c8 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:48:57.559825 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 601/1199] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 03:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:48:57.566163 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c85b68-4133-4925-abf6-a36b8f00010c None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:48:57.566600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c85b68-4133-4925-abf6-a36b8f00010c None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:48:57.566705 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c85b68-4133-4925-abf6-a36b8f00010c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:48:57.586655 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-50c85b68-4133-4925-abf6-a36b8f00010c None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:48:57.587893 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-50c85b68-4133-4925-abf6-a36b8f00010c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:48:57.595887 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 599/1200] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 03:48:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:48:57.750446 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190bd5e-d706-4dfb-a25f-8ecdd2b962c8 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:48:57.764626 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190bd5e-d706-4dfb-a25f-8ecdd2b962c8 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:48:57.770120 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f033385a-6416-4bc2-8445-909af53626ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:48:57.770715 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033385a-6416-4bc2-8445-909af53626ff service nova] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:48:57.771022 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033385a-6416-4bc2-8445-909af53626ff service nova] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:48:57.771319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f033385a-6416-4bc2-8445-909af53626ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:48:57.780031 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f033385a-6416-4bc2-8445-909af53626ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:48:57.782182 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f033385a-6416-4bc2-8445-909af53626ff service nova] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:48:57.792193 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 602/1201] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:48:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:48:58.856850 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7987a-fd34-444a-b1d3-669a2e9f5cc4 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:48:58.857421 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7987a-fd34-444a-b1d3-669a2e9f5cc4 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:48:58.857421 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e7987a-fd34-444a-b1d3-669a2e9f5cc4 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:48:58.858768 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 600/1202] 127.0.0.1 () {54 vars in 901 bytes} [Mon Oct 17 03:48:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:48:58.864918 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6f1d3-dccf-43c5-966d-6e967d768180 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:48:58.865364 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6f1d3-dccf-43c5-966d-6e967d768180 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:48:58.865478 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c6f1d3-dccf-43c5-966d-6e967d768180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:48:58.885592 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-35c6f1d3-dccf-43c5-966d-6e967d768180 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:48:58.886804 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-35c6f1d3-dccf-43c5-966d-6e967d768180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:48:58.894793 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 603/1203] 127.0.0.1 () {58 vars in 998 bytes} [Mon Oct 17 03:48:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:48:59.047498 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7987a-fd34-444a-b1d3-669a2e9f5cc4 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:48:59.061880 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e7987a-fd34-444a-b1d3-669a2e9f5cc4 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:48:59.067185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21425c65-3fdf-440b-9202-5e32fadf8032 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:48:59.067798 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21425c65-3fdf-440b-9202-5e32fadf8032 service nova] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:48:59.068078 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21425c65-3fdf-440b-9202-5e32fadf8032 service nova] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:48:59.068347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21425c65-3fdf-440b-9202-5e32fadf8032 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:48:59.076756 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21425c65-3fdf-440b-9202-5e32fadf8032 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:48:59.078311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21425c65-3fdf-440b-9202-5e32fadf8032 service nova] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:48:59.087633 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 601/1204] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:48:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:49:02.989341 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfc772-4ccf-43ab-872d-cebc156f967d None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:49:02.989879 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfc772-4ccf-43ab-872d-cebc156f967d None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:49:02.989879 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdfc772-4ccf-43ab-872d-cebc156f967d None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:49:02.991233 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 604/1205] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:49:02.996810 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9318e0-3657-4d3f-8cd7-1327a969e4c2 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:49:02.997254 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9318e0-3657-4d3f-8cd7-1327a969e4c2 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:49:02.997358 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9318e0-3657-4d3f-8cd7-1327a969e4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:49:03.016515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-3a9318e0-3657-4d3f-8cd7-1327a969e4c2 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:49:03.017721 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-3a9318e0-3657-4d3f-8cd7-1327a969e4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:49:03.026001 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 602/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:49:03.317077 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdfc772-4ccf-43ab-872d-cebc156f967d None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:49:03.318779 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-6cdfc772-4ccf-43ab-872d-cebc156f967d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:49:03.395656 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdfc772-4ccf-43ab-872d-cebc156f967d None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:49:03.399790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec71bf39-569a-4bd7-94ad-7e18fb7cf491 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:49:03.400389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71bf39-569a-4bd7-94ad-7e18fb7cf491 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:49:03.400669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71bf39-569a-4bd7-94ad-7e18fb7cf491 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:49:03.400940 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71bf39-569a-4bd7-94ad-7e18fb7cf491 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:49:03.409736 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71bf39-569a-4bd7-94ad-7e18fb7cf491 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:49:03.411153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec71bf39-569a-4bd7-94ad-7e18fb7cf491 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:49:03.420568 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 605/1207] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:49:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:49:05.546663 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0222d1-8e82-49b8-ae12-21d120a40e95 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:49:05.547322 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0222d1-8e82-49b8-ae12-21d120a40e95 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:49:05.547322 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0222d1-8e82-49b8-ae12-21d120a40e95 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:49:05.548627 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 603/1208] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:49:05.553710 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa391952-feed-400a-9655-50302dea3a75 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:49:05.554162 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa391952-feed-400a-9655-50302dea3a75 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:49:05.554258 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa391952-feed-400a-9655-50302dea3a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:49:05.574111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-fa391952-feed-400a-9655-50302dea3a75 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:49:05.575314 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-fa391952-feed-400a-9655-50302dea3a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:49:05.583047 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 606/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:49:05.871344 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0222d1-8e82-49b8-ae12-21d120a40e95 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:49:05.881874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0222d1-8e82-49b8-ae12-21d120a40e95 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:49:05.885611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997cd4c4-29d6-46ba-ab05-7f665c84655e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:49:05.886199 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cd4c4-29d6-46ba-ab05-7f665c84655e service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:49:05.886476 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cd4c4-29d6-46ba-ab05-7f665c84655e service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:49:05.886747 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997cd4c4-29d6-46ba-ab05-7f665c84655e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:49:05.895356 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997cd4c4-29d6-46ba-ab05-7f665c84655e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:49:05.896799 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997cd4c4-29d6-46ba-ab05-7f665c84655e service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:49:05.906439 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 604/1210] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:49:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:49:09.429743 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33058e-b052-4d95-b535-746ec8ca0fe4 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:49:09.430219 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33058e-b052-4d95-b535-746ec8ca0fe4 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:49:09.430340 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33058e-b052-4d95-b535-746ec8ca0fe4 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:49:09.431645 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 607/1211] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:49:09.436850 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266ded5-8c90-42bd-b11b-b3d40115dcc1 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:49:09.437279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266ded5-8c90-42bd-b11b-b3d40115dcc1 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:49:09.437469 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a266ded5-8c90-42bd-b11b-b3d40115dcc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:49:09.456844 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-a266ded5-8c90-42bd-b11b-b3d40115dcc1 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:49:09.458017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-a266ded5-8c90-42bd-b11b-b3d40115dcc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:49:09.465985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 605/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:49:09.776082 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec33058e-b052-4d95-b535-746ec8ca0fe4 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:49:09.777741 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-ec33058e-b052-4d95-b535-746ec8ca0fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:49:09.841624 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:49:09.855245 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec33058e-b052-4d95-b535-746ec8ca0fe4 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:49:09.860909 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6ce99-4b84-446c-8ef0-4c1e87e76f93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:49:09.860909 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6ce99-4b84-446c-8ef0-4c1e87e76f93 service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:49:09.860909 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6ce99-4b84-446c-8ef0-4c1e87e76f93 service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:49:09.860909 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6ce99-4b84-446c-8ef0-4c1e87e76f93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:49:09.870339 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c6ce99-4b84-446c-8ef0-4c1e87e76f93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:49:09.871804 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c6ce99-4b84-446c-8ef0-4c1e87e76f93 service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:49:09.880842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 608/1213] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:49:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:49:11.992253 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180040e0-d8ea-49a5-83ae-815e5a462758 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:49:11.992817 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180040e0-d8ea-49a5-83ae-815e5a462758 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:49:11.992817 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180040e0-d8ea-49a5-83ae-815e5a462758 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:49:11.994201 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 606/1214] 127.0.0.1 () {54 vars in 917 bytes} [Mon Oct 17 03:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:49:11.999303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372233f9-0b8c-4627-98ee-d26ddcf2862e None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:49:11.999770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372233f9-0b8c-4627-98ee-d26ddcf2862e None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:49:11.999889 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372233f9-0b8c-4627-98ee-d26ddcf2862e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:49:12.020057 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-372233f9-0b8c-4627-98ee-d26ddcf2862e None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:49:12.021257 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-372233f9-0b8c-4627-98ee-d26ddcf2862e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:49:12.029377 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 609/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Oct 17 03:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:49:12.325095 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180040e0-d8ea-49a5-83ae-815e5a462758 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:49:12.326774 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-180040e0-d8ea-49a5-83ae-815e5a462758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:49:12.334296 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=84 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:49:12.341220 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180040e0-d8ea-49a5-83ae-815e5a462758 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:49:12.344835 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210ac01-0c46-4958-abeb-a1f3c65cc451 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:49:12.345430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210ac01-0c46-4958-abeb-a1f3c65cc451 service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:49:12.345707 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210ac01-0c46-4958-abeb-a1f3c65cc451 service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:49:12.345974 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210ac01-0c46-4958-abeb-a1f3c65cc451 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:49:12.354836 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210ac01-0c46-4958-abeb-a1f3c65cc451 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:49:12.356282 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2210ac01-0c46-4958-abeb-a1f3c65cc451 service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:49:12.365380 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 607/1216] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:49:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:49:22.015114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=11 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:49:22.380045 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=5 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:37.626123 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e10db-4adb-4c6d-9073-c5bb6337212c None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:37.628306 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e10db-4adb-4c6d-9073-c5bb6337212c None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:37.628306 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374e10db-4adb-4c6d-9073-c5bb6337212c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:37.631999 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164a283-ee10-4bb0-a296-a3d1d4496397 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:37.633013 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164a283-ee10-4bb0-a296-a3d1d4496397 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:37.633103 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164a283-ee10-4bb0-a296-a3d1d4496397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:37.661788 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-374e10db-4adb-4c6d-9073-c5bb6337212c None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:37.663021 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-374e10db-4adb-4c6d-9073-c5bb6337212c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:37.676879 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-7164a283-ee10-4bb0-a296-a3d1d4496397 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:37.678129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-7164a283-ee10-4bb0-a296-a3d1d4496397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:37.683578 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 610/1217] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:37.698220 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 608/1218] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:37.704254 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cede1f-344f-4f9d-8abc-9a488579b115 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:37.704534 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cede1f-344f-4f9d-8abc-9a488579b115 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:37.704817 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cede1f-344f-4f9d-8abc-9a488579b115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:37.709956 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8949f-3e34-46ef-ac62-3bb530b333bd None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:37.710378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8949f-3e34-46ef-ac62-3bb530b333bd None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:37.711363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8949f-3e34-46ef-ac62-3bb530b333bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:37.739461 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-43cede1f-344f-4f9d-8abc-9a488579b115 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:37.740869 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-9ad8949f-3e34-46ef-ac62-3bb530b333bd None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:37.742374 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-43cede1f-344f-4f9d-8abc-9a488579b115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:37.743957 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-9ad8949f-3e34-46ef-ac62-3bb530b333bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:37.755297 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 609/1219] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:37.756964 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2d8fc-591c-45b1-b99c-469f3e7e44d2 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:37.757991 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2d8fc-591c-45b1-b99c-469f3e7e44d2 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:37.758476 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2d8fc-591c-45b1-b99c-469f3e7e44d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:37.777875 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 611/1220] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:37.783455 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3461e-810e-4283-836d-616687e56e90 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:37.783455 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3461e-810e-4283-836d-616687e56e90 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:37.783455 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b3461e-810e-4283-836d-616687e56e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:37.785059 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-25b2d8fc-591c-45b1-b99c-469f3e7e44d2 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:37.786505 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-25b2d8fc-591c-45b1-b99c-469f3e7e44d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:37.800007 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 610/1221] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:37.805685 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d2d38-0bba-415c-adc9-c5f9b7b888cb None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:37.805968 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d2d38-0bba-415c-adc9-c5f9b7b888cb None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:37.806289 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1d2d38-0bba-415c-adc9-c5f9b7b888cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:37.810922 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-07b3461e-810e-4283-836d-616687e56e90 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:37.812147 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-07b3461e-810e-4283-836d-616687e56e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:37.821554 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 612/1222] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:37.822780 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c38d26-50cb-4817-954d-6064fae15d6d None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:37.823304 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c38d26-50cb-4817-954d-6064fae15d6d None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:37.823591 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c38d26-50cb-4817-954d-6064fae15d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:37.828132 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-cc1d2d38-0bba-415c-adc9-c5f9b7b888cb None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:37.829323 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-cc1d2d38-0bba-415c-adc9-c5f9b7b888cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:37.837551 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 611/1223] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:37.838462 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d2d38-0bba-415c-adc9-c5f9b7b888cb None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:37.844736 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-a3c38d26-50cb-4817-954d-6064fae15d6d None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:37.845964 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-a3c38d26-50cb-4817-954d-6064fae15d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:37.847879 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1d2d38-0bba-415c-adc9-c5f9b7b888cb None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:37.852291 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3d90f-3429-437d-af80-ca97804f739c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:37.852881 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3d90f-3429-437d-af80-ca97804f739c admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:37.853156 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3d90f-3429-437d-af80-ca97804f739c admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:37.853425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3d90f-3429-437d-af80-ca97804f739c admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:37.853914 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 613/1224] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:37.855083 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb27cd-8590-45dd-a37c-0286c3a7446e None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:37.855373 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb27cd-8590-45dd-a37c-0286c3a7446e None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:37.855647 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb27cd-8590-45dd-a37c-0286c3a7446e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:37.858776 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3d90f-3429-437d-af80-ca97804f739c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:50:37.859219 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3d90f-3429-437d-af80-ca97804f739c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:37.860536 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: WARNING py.warnings [None req-1cf3d90f-3429-437d-af80-ca97804f739c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 17 03:50:37.860536 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: warnings.warn(msg) Oct 17 03:50:37.860536 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: Oct 17 03:50:37.861430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf3d90f-3429-437d-af80-ca97804f739c admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:37.874427 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 612/1225] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:50:37.875528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-3fdb27cd-8590-45dd-a37c-0286c3a7446e None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:37.876022 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aee296-7207-40a2-abed-837d37d85f03 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:37.876315 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aee296-7207-40a2-abed-837d37d85f03 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:37.876591 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aee296-7207-40a2-abed-837d37d85f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:37.876761 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-3fdb27cd-8590-45dd-a37c-0286c3a7446e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:37.886135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 614/1226] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:37.886311 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb27cd-8590-45dd-a37c-0286c3a7446e None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:37.895978 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb27cd-8590-45dd-a37c-0286c3a7446e None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:37.896605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-a3aee296-7207-40a2-abed-837d37d85f03 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:37.897853 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-a3aee296-7207-40a2-abed-837d37d85f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:37.900908 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7a034-aa39-41d5-a1c0-8ce9cec14a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:37.901514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7a034-aa39-41d5-a1c0-8ce9cec14a60 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:37.901796 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7a034-aa39-41d5-a1c0-8ce9cec14a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:37.902070 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7a034-aa39-41d5-a1c0-8ce9cec14a60 admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:37.902710 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd7a034-aa39-41d5-a1c0-8ce9cec14a60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:50:37.903043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd7a034-aa39-41d5-a1c0-8ce9cec14a60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:37.904654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd7a034-aa39-41d5-a1c0-8ce9cec14a60 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:37.905848 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 613/1227] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:37.907270 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527e8be-30a4-4f41-b3d5-6c603f6c7f24 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:37.907557 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527e8be-30a4-4f41-b3d5-6c603f6c7f24 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:37.907861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527e8be-30a4-4f41-b3d5-6c603f6c7f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:37.923011 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 615/1228] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:50:37.924506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d64199-0481-4e7e-8d36-635ccd6e41a9 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:37.924940 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d64199-0481-4e7e-8d36-635ccd6e41a9 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:37.925363 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d64199-0481-4e7e-8d36-635ccd6e41a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:37.926843 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-5527e8be-30a4-4f41-b3d5-6c603f6c7f24 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:37.928055 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-5527e8be-30a4-4f41-b3d5-6c603f6c7f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:37.936734 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 614/1229] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:37.937929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c5aae-bd56-453e-817e-cbac0f3f8c7f None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:37.938215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c5aae-bd56-453e-817e-cbac0f3f8c7f None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:37.938480 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c5aae-bd56-453e-817e-cbac0f3f8c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:37.952345 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-75d64199-0481-4e7e-8d36-635ccd6e41a9 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:37.953551 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-75d64199-0481-4e7e-8d36-635ccd6e41a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:37.958169 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-d96c5aae-bd56-453e-817e-cbac0f3f8c7f None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:37.959307 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-d96c5aae-bd56-453e-817e-cbac0f3f8c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:37.962079 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 616/1230] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:37.963267 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9670ba-5ca2-47cc-965b-4b83a40ffbb9 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:37.963557 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9670ba-5ca2-47cc-965b-4b83a40ffbb9 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:37.963865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9670ba-5ca2-47cc-965b-4b83a40ffbb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:37.969884 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 615/1231] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:37.970827 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c5aae-bd56-453e-817e-cbac0f3f8c7f None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:37.980499 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c5aae-bd56-453e-817e-cbac0f3f8c7f None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:37.985221 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c656187-732f-4fec-8e6e-b431ecf6e9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:37.985542 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-4f9670ba-5ca2-47cc-965b-4b83a40ffbb9 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:37.985826 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c656187-732f-4fec-8e6e-b431ecf6e9fc admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:37.986083 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c656187-732f-4fec-8e6e-b431ecf6e9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:37.986351 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c656187-732f-4fec-8e6e-b431ecf6e9fc admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:37.986729 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-4f9670ba-5ca2-47cc-965b-4b83a40ffbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:37.986972 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c656187-732f-4fec-8e6e-b431ecf6e9fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:37.988885 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c656187-732f-4fec-8e6e-b431ecf6e9fc admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:37.995032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 617/1232] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:37.995839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9670ba-5ca2-47cc-965b-4b83a40ffbb9 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:38.005319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9670ba-5ca2-47cc-965b-4b83a40ffbb9 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:38.010409 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b9724-6989-44b8-8390-4be6e6655b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:38.010995 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b9724-6989-44b8-8390-4be6e6655b74 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:38.011276 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b9724-6989-44b8-8390-4be6e6655b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:38.011550 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b9724-6989-44b8-8390-4be6e6655b74 admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:38.012220 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b9724-6989-44b8-8390-4be6e6655b74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:50:38.012520 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b9724-6989-44b8-8390-4be6e6655b74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:38.014095 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b9724-6989-44b8-8390-4be6e6655b74 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:38.025620 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 618/1233] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:50:38.032781 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a653b-57ab-44b7-9535-3cd141ca89a7 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:38.032781 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a653b-57ab-44b7-9535-3cd141ca89a7 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:38.032781 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a653b-57ab-44b7-9535-3cd141ca89a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:38.046060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-cb3a653b-57ab-44b7-9535-3cd141ca89a7 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:38.047230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-cb3a653b-57ab-44b7-9535-3cd141ca89a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:38.055433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 619/1234] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:38.056362 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3a653b-57ab-44b7-9535-3cd141ca89a7 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:38.065372 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3a653b-57ab-44b7-9535-3cd141ca89a7 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:38.070601 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528c19f-2e6f-43d4-9e21-7758b84b84e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:38.071160 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528c19f-2e6f-43d4-9e21-7758b84b84e4 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:38.071438 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528c19f-2e6f-43d4-9e21-7758b84b84e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:38.071707 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528c19f-2e6f-43d4-9e21-7758b84b84e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:38.072414 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2528c19f-2e6f-43d4-9e21-7758b84b84e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:38.073398 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: WARNING py.warnings [None req-2528c19f-2e6f-43d4-9e21-7758b84b84e4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 03:50:38.073398 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: warnings.warn(msg) Oct 17 03:50:38.073398 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: Oct 17 03:50:38.074558 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2528c19f-2e6f-43d4-9e21-7758b84b84e4 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:38.098508 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 616/1235] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:50:37 2022] POST /identity/v3/projects => generated 373 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:50:38.100010 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cc9e2-c732-435a-80e4-c2a75c4bcd0d None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:38.100296 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cc9e2-c732-435a-80e4-c2a75c4bcd0d None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:38.100569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cc9e2-c732-435a-80e4-c2a75c4bcd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:38.120000 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-e54cc9e2-c732-435a-80e4-c2a75c4bcd0d None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:38.121232 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-e54cc9e2-c732-435a-80e4-c2a75c4bcd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:38.161700 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 620/1236] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:50:38 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:50:38.162875 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528c19f-2e6f-43d4-9e21-7758b84b84e4 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:38.184098 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 617/1237] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:38.185245 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54cc9e2-c732-435a-80e4-c2a75c4bcd0d None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:38.197169 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54cc9e2-c732-435a-80e4-c2a75c4bcd0d None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:38.224359 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528c19f-2e6f-43d4-9e21-7758b84b84e4 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:38.229705 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a9ad64-afd8-4b41-b3be-2548a76fe5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:38.230504 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a9ad64-afd8-4b41-b3be-2548a76fe5a6 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:38.231599 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a9ad64-afd8-4b41-b3be-2548a76fe5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:38.231927 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a9ad64-afd8-4b41-b3be-2548a76fe5a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:38.232610 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a9ad64-afd8-4b41-b3be-2548a76fe5a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:50:38.232916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a9ad64-afd8-4b41-b3be-2548a76fe5a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:38.234535 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a9ad64-afd8-4b41-b3be-2548a76fe5a6 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:38.246793 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 621/1238] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:50:38.247593 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a9ad64-afd8-4b41-b3be-2548a76fe5a6 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:38.255403 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b557b0-9cad-4341-b29a-c9bf9d22e4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:38.256054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b557b0-9cad-4341-b29a-c9bf9d22e4e5 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:38.256338 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b557b0-9cad-4341-b29a-c9bf9d22e4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:38.256614 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b557b0-9cad-4341-b29a-c9bf9d22e4e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:38.257265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b557b0-9cad-4341-b29a-c9bf9d22e4e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:50:38.257550 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a9ad64-afd8-4b41-b3be-2548a76fe5a6 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:38.257652 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b557b0-9cad-4341-b29a-c9bf9d22e4e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:38.259149 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b557b0-9cad-4341-b29a-c9bf9d22e4e5 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:38.262138 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ebae4-9207-4c8e-9016-4f083009bef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:38.262720 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ebae4-9207-4c8e-9016-4f083009bef4 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:38.262999 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ebae4-9207-4c8e-9016-4f083009bef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:38.263272 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ebae4-9207-4c8e-9016-4f083009bef4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:38.263931 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ebae4-9207-4c8e-9016-4f083009bef4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:50:38.264238 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ebae4-9207-4c8e-9016-4f083009bef4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:38.265764 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ebae4-9207-4c8e-9016-4f083009bef4 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:38.273095 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 618/1239] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:50:38.273095 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b557b0-9cad-4341-b29a-c9bf9d22e4e5 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:38.277579 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 622/1240] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:50:38.278383 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ebae4-9207-4c8e-9016-4f083009bef4 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:38.280531 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:38.284340 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b557b0-9cad-4341-b29a-c9bf9d22e4e5 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:38.288205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ebae4-9207-4c8e-9016-4f083009bef4 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:38.292618 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2520d83-3628-417a-b98e-29711b247193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:38.292618 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2520d83-3628-417a-b98e-29711b247193 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:38.292618 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2520d83-3628-417a-b98e-29711b247193 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:38.292618 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2520d83-3628-417a-b98e-29711b247193 admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:38.292999 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2520d83-3628-417a-b98e-29711b247193 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:38.293321 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58a76b-1ec8-4833-a8f8-60e335623e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:38.293626 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2520d83-3628-417a-b98e-29711b247193 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:38.293907 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58a76b-1ec8-4833-a8f8-60e335623e0b admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:38.294188 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58a76b-1ec8-4833-a8f8-60e335623e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:38.294462 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58a76b-1ec8-4833-a8f8-60e335623e0b admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:38.295099 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb58a76b-1ec8-4833-a8f8-60e335623e0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:50:38.295398 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb58a76b-1ec8-4833-a8f8-60e335623e0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:38.296979 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb58a76b-1ec8-4833-a8f8-60e335623e0b admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:38.309119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 623/1241] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:50:38.310154 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58a76b-1ec8-4833-a8f8-60e335623e0b admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:38.320462 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58a76b-1ec8-4833-a8f8-60e335623e0b admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:38.325049 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c6d78-9cfb-41f2-9c69-81fd3d501e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:38.325621 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c6d78-9cfb-41f2-9c69-81fd3d501e83 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:38.325899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c6d78-9cfb-41f2-9c69-81fd3d501e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:38.326167 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24c6d78-9cfb-41f2-9c69-81fd3d501e83 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:38.326785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c6d78-9cfb-41f2-9c69-81fd3d501e83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:38.328730 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24c6d78-9cfb-41f2-9c69-81fd3d501e83 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:38.366398 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 619/1242] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:50:38 2022] POST /identity/v3/projects => generated 379 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:50:38.367758 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2520d83-3628-417a-b98e-29711b247193 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:38.417400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 624/1243] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:50:38 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:50:38.418592 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c6d78-9cfb-41f2-9c69-81fd3d501e83 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:38.428909 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24c6d78-9cfb-41f2-9c69-81fd3d501e83 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:38.431966 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2520d83-3628-417a-b98e-29711b247193 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:38.434113 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309701d7-47ea-4ab3-8d6e-99e71c98a0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:38.434700 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309701d7-47ea-4ab3-8d6e-99e71c98a0e4 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:38.434980 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309701d7-47ea-4ab3-8d6e-99e71c98a0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:38.435254 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309701d7-47ea-4ab3-8d6e-99e71c98a0e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:38.435923 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309701d7-47ea-4ab3-8d6e-99e71c98a0e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:50:38.436227 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309701d7-47ea-4ab3-8d6e-99e71c98a0e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:38.436545 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73005bd-71a1-411c-aed9-68174ed7a485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:38.437189 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73005bd-71a1-411c-aed9-68174ed7a485 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:38.437465 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73005bd-71a1-411c-aed9-68174ed7a485 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:38.437738 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73005bd-71a1-411c-aed9-68174ed7a485 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:38.437855 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309701d7-47ea-4ab3-8d6e-99e71c98a0e4 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:38.438365 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73005bd-71a1-411c-aed9-68174ed7a485 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:38.439388 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: WARNING py.warnings [None req-f73005bd-71a1-411c-aed9-68174ed7a485 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 03:50:38.439388 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: warnings.warn(msg) Oct 17 03:50:38.439388 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: Oct 17 03:50:38.440225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:38.440634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73005bd-71a1-411c-aed9-68174ed7a485 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:38.450279 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 625/1244] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:50:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:50:38.458294 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309701d7-47ea-4ab3-8d6e-99e71c98a0e4 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:38.463702 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309701d7-47ea-4ab3-8d6e-99e71c98a0e4 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:38.468453 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8093ebe-ea01-420c-a1e8-1ac8f9919468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:38.469085 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8093ebe-ea01-420c-a1e8-1ac8f9919468 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:38.469373 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8093ebe-ea01-420c-a1e8-1ac8f9919468 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:38.469677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8093ebe-ea01-420c-a1e8-1ac8f9919468 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:38.470360 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8093ebe-ea01-420c-a1e8-1ac8f9919468 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:38.472376 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8093ebe-ea01-420c-a1e8-1ac8f9919468 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:38.548548 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 620/1245] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:50:38 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:50:38.549475 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73005bd-71a1-411c-aed9-68174ed7a485 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:38.559020 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73005bd-71a1-411c-aed9-68174ed7a485 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:38.563805 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8276cf-ef58-4491-a370-14bf616bcff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:38.564394 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8276cf-ef58-4491-a370-14bf616bcff5 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:38.564689 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8276cf-ef58-4491-a370-14bf616bcff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:38.565000 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8276cf-ef58-4491-a370-14bf616bcff5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:38.565592 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8276cf-ef58-4491-a370-14bf616bcff5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:38.567024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 626/1246] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:50:38 2022] POST /identity/v3/projects => generated 393 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:50:38.567486 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8276cf-ef58-4491-a370-14bf616bcff5 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:38.567701 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8093ebe-ea01-420c-a1e8-1ac8f9919468 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:38.635457 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8093ebe-ea01-420c-a1e8-1ac8f9919468 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:38.640346 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a085cb2-0340-4708-8c2d-5108282ac735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:38.641009 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a085cb2-0340-4708-8c2d-5108282ac735 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:38.641287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a085cb2-0340-4708-8c2d-5108282ac735 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:38.641562 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a085cb2-0340-4708-8c2d-5108282ac735 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:38.642210 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a085cb2-0340-4708-8c2d-5108282ac735 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:38.644178 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a085cb2-0340-4708-8c2d-5108282ac735 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:38.727503 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 627/1247] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:50:38 2022] POST /identity/v3/projects => generated 383 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:50:38.728361 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a085cb2-0340-4708-8c2d-5108282ac735 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:38.786417 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a085cb2-0340-4708-8c2d-5108282ac735 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:38.790193 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 621/1248] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:50:38 2022] POST /identity/v3/projects => generated 367 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:50:38.790931 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8276cf-ef58-4491-a370-14bf616bcff5 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:38.793620 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b09e37a-e99f-4580-a804-48994740fa6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:38.794251 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09e37a-e99f-4580-a804-48994740fa6e admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:38.794524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09e37a-e99f-4580-a804-48994740fa6e admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:38.794801 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09e37a-e99f-4580-a804-48994740fa6e admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:38.795441 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b09e37a-e99f-4580-a804-48994740fa6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:38.797487 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b09e37a-e99f-4580-a804-48994740fa6e admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:38.853413 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8276cf-ef58-4491-a370-14bf616bcff5 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:38.857984 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c71685-d314-4fc8-9011-fa7ce8302ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:38.858589 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c71685-d314-4fc8-9011-fa7ce8302ffb admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:38.858871 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c71685-d314-4fc8-9011-fa7ce8302ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:38.859145 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c71685-d314-4fc8-9011-fa7ce8302ffb admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:38.859802 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c71685-d314-4fc8-9011-fa7ce8302ffb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:38.861782 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c71685-d314-4fc8-9011-fa7ce8302ffb admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:38.869864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 628/1249] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:50:38 2022] POST /identity/v3/projects => generated 357 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:50:38.870611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b09e37a-e99f-4580-a804-48994740fa6e admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:38.915364 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:38.930873 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b09e37a-e99f-4580-a804-48994740fa6e admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:38.935563 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad80a0-786e-4181-a261-34aa154ce915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:38.945205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad80a0-786e-4181-a261-34aa154ce915 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:38.945205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad80a0-786e-4181-a261-34aa154ce915 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:38.945205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ad80a0-786e-4181-a261-34aa154ce915 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:38.945205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad80a0-786e-4181-a261-34aa154ce915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:38.945205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ad80a0-786e-4181-a261-34aa154ce915 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:38.948897 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 629/1250] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:50:38 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:50:38.949664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad80a0-786e-4181-a261-34aa154ce915 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:38.959137 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ad80a0-786e-4181-a261-34aa154ce915 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:38.964428 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d2e55-6365-40d2-b124-da6848f54d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:38.964996 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d2e55-6365-40d2-b124-da6848f54d0b admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:38.965283 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d2e55-6365-40d2-b124-da6848f54d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:38.965563 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d2e55-6365-40d2-b124-da6848f54d0b admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:38.966183 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d2e55-6365-40d2-b124-da6848f54d0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:38.968048 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d2e55-6365-40d2-b124-da6848f54d0b admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:38.971723 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 622/1251] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:50:38 2022] POST /identity/v3/users => generated 290 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:50:38.971723 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c71685-d314-4fc8-9011-fa7ce8302ffb admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:38.981772 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c71685-d314-4fc8-9011-fa7ce8302ffb admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:38.986383 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c1d3b-63a6-4284-9e68-fe38a8ac80ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:38.987035 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c1d3b-63a6-4284-9e68-fe38a8ac80ae admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:38.987313 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c1d3b-63a6-4284-9e68-fe38a8ac80ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:38.987578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c1d3b-63a6-4284-9e68-fe38a8ac80ae admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:38.988245 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c1d3b-63a6-4284-9e68-fe38a8ac80ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:38.989812 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c1d3b-63a6-4284-9e68-fe38a8ac80ae admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:38.999975 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 623/1252] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:50:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:50:39.000859 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c1d3b-63a6-4284-9e68-fe38a8ac80ae admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:39.010552 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c1d3b-63a6-4284-9e68-fe38a8ac80ae admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:39.015128 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0bd328-67d2-4393-a0d3-76cc77aafa62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:39.015688 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0bd328-67d2-4393-a0d3-76cc77aafa62 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:39.015996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0bd328-67d2-4393-a0d3-76cc77aafa62 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:39.016268 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0bd328-67d2-4393-a0d3-76cc77aafa62 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:39.016865 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0bd328-67d2-4393-a0d3-76cc77aafa62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:39.018686 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0bd328-67d2-4393-a0d3-76cc77aafa62 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:39.076159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 630/1253] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:50:38 2022] POST /identity/v3/projects => generated 389 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:50:39.077036 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d2e55-6365-40d2-b124-da6848f54d0b admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:39.109194 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 624/1254] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:50:38 2022] POST /identity/v3/users => generated 297 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:50:39.110061 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0bd328-67d2-4393-a0d3-76cc77aafa62 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:39.119774 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0bd328-67d2-4393-a0d3-76cc77aafa62 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:39.127624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:39.139124 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d2e55-6365-40d2-b124-da6848f54d0b admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:39.143642 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea504f-4e8f-4458-8954-b5df76c58c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:39.144288 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea504f-4e8f-4458-8954-b5df76c58c8f admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:39.144572 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea504f-4e8f-4458-8954-b5df76c58c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:39.144841 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea504f-4e8f-4458-8954-b5df76c58c8f admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:39.145465 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea504f-4e8f-4458-8954-b5df76c58c8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:39.147399 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ea504f-4e8f-4458-8954-b5df76c58c8f admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:39.176011 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77514555-65ab-4b0f-9aef-8ffaf82d9839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:39.176626 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77514555-65ab-4b0f-9aef-8ffaf82d9839 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:39.177114 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77514555-65ab-4b0f-9aef-8ffaf82d9839 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:39.177757 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77514555-65ab-4b0f-9aef-8ffaf82d9839 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:39.178454 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77514555-65ab-4b0f-9aef-8ffaf82d9839 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:39.180463 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77514555-65ab-4b0f-9aef-8ffaf82d9839 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:39.262785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 631/1255] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:50:39 2022] POST /identity/v3/users => generated 292 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:50:39.263788 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea504f-4e8f-4458-8954-b5df76c58c8f admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:39.273763 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea504f-4e8f-4458-8954-b5df76c58c8f admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:39.278381 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f6c8ce-7b6c-4f77-8eb3-e0ad0fd3ab71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:39.278943 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6c8ce-7b6c-4f77-8eb3-e0ad0fd3ab71 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:39.279221 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6c8ce-7b6c-4f77-8eb3-e0ad0fd3ab71 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:39.279493 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6c8ce-7b6c-4f77-8eb3-e0ad0fd3ab71 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:39.280144 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f6c8ce-7b6c-4f77-8eb3-e0ad0fd3ab71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:39.282032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f6c8ce-7b6c-4f77-8eb3-e0ad0fd3ab71 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:39.291470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 625/1256] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:50:39 2022] POST /identity/v3/users => generated 284 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:50:39.294866 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77514555-65ab-4b0f-9aef-8ffaf82d9839 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:39.302416 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77514555-65ab-4b0f-9aef-8ffaf82d9839 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:39.307490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f235c73-5ec1-4318-ab67-051ba4c62426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:39.308133 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f235c73-5ec1-4318-ab67-051ba4c62426 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:39.308415 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f235c73-5ec1-4318-ab67-051ba4c62426 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:39.308690 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f235c73-5ec1-4318-ab67-051ba4c62426 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:39.309314 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f235c73-5ec1-4318-ab67-051ba4c62426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:39.310888 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f235c73-5ec1-4318-ab67-051ba4c62426 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:39.321286 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 626/1257] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:50:39.323492 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f235c73-5ec1-4318-ab67-051ba4c62426 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:39.331132 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f235c73-5ec1-4318-ab67-051ba4c62426 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:39.335912 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc9b8c-44af-4c36-a106-b2e95224c951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:39.336459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc9b8c-44af-4c36-a106-b2e95224c951 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:39.336743 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc9b8c-44af-4c36-a106-b2e95224c951 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:39.337022 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc9b8c-44af-4c36-a106-b2e95224c951 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:39.337623 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc9b8c-44af-4c36-a106-b2e95224c951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:39.339115 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc9b8c-44af-4c36-a106-b2e95224c951 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:39.349290 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 627/1258] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:50:39.353892 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc9b8c-44af-4c36-a106-b2e95224c951 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:39.359011 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc9b8c-44af-4c36-a106-b2e95224c951 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:39.363464 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b945b-a0ce-4528-9026-9758ee6ee543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:39.364013 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b945b-a0ce-4528-9026-9758ee6ee543 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:39.364296 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b945b-a0ce-4528-9026-9758ee6ee543 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:39.364571 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b945b-a0ce-4528-9026-9758ee6ee543 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:39.365170 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37b945b-a0ce-4528-9026-9758ee6ee543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:39.367157 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37b945b-a0ce-4528-9026-9758ee6ee543 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:39.376969 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 628/1259] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:50:39.380920 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 632/1260] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:50:39 2022] POST /identity/v3/users => generated 279 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:50:39.381027 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b945b-a0ce-4528-9026-9758ee6ee543 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:39.381106 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f6c8ce-7b6c-4f77-8eb3-e0ad0fd3ab71 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:39.388605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b945b-a0ce-4528-9026-9758ee6ee543 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:39.390173 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f6c8ce-7b6c-4f77-8eb3-e0ad0fd3ab71 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:39.393473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa498d4-a33a-4190-a922-fc27aead01ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:39.394090 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa498d4-a33a-4190-a922-fc27aead01ba admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:39.394367 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa498d4-a33a-4190-a922-fc27aead01ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:39.394640 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa498d4-a33a-4190-a922-fc27aead01ba admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:39.394919 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c2932-8ca0-40f2-b29e-20d5ba15dd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:39.395270 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa498d4-a33a-4190-a922-fc27aead01ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:39.395516 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c2932-8ca0-40f2-b29e-20d5ba15dd82 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:39.395823 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c2932-8ca0-40f2-b29e-20d5ba15dd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:39.396099 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c2932-8ca0-40f2-b29e-20d5ba15dd82 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:39.396732 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648c2932-8ca0-40f2-b29e-20d5ba15dd82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:39.397187 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa498d4-a33a-4190-a922-fc27aead01ba admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:39.398326 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648c2932-8ca0-40f2-b29e-20d5ba15dd82 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:39.409022 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 633/1261] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:50:39.409624 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c2932-8ca0-40f2-b29e-20d5ba15dd82 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:39.418927 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648c2932-8ca0-40f2-b29e-20d5ba15dd82 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:39.423451 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4af62-f851-48e3-b058-f19387a04f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:39.424045 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4af62-f851-48e3-b058-f19387a04f79 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:39.424322 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4af62-f851-48e3-b058-f19387a04f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:39.424593 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4af62-f851-48e3-b058-f19387a04f79 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:39.425527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb4af62-f851-48e3-b058-f19387a04f79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:39.426730 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb4af62-f851-48e3-b058-f19387a04f79 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:39.436702 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 634/1262] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:50:39.437450 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4af62-f851-48e3-b058-f19387a04f79 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:39.446896 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb4af62-f851-48e3-b058-f19387a04f79 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:39.451300 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c136135-f7da-4a78-b462-35027e1ba79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:39.451861 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c136135-f7da-4a78-b462-35027e1ba79f admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:39.452141 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c136135-f7da-4a78-b462-35027e1ba79f admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:39.452412 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c136135-f7da-4a78-b462-35027e1ba79f admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:39.453002 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c136135-f7da-4a78-b462-35027e1ba79f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:39.454476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c136135-f7da-4a78-b462-35027e1ba79f admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:39.467377 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 635/1263] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:50:39.467377 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c136135-f7da-4a78-b462-35027e1ba79f admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:39.474670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c136135-f7da-4a78-b462-35027e1ba79f admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:39.479358 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509367f-35d8-4d22-9c59-e5a4f07e750f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:39.480010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509367f-35d8-4d22-9c59-e5a4f07e750f admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:39.480277 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509367f-35d8-4d22-9c59-e5a4f07e750f admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:39.480548 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509367f-35d8-4d22-9c59-e5a4f07e750f admin admin] PATH_INFO: `/v3/projects/f8f11d4419f946b9aa0accbc6a219488/users/e2a2f995befa475583e672a9e89ed281/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:39.488562 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 629/1264] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:50:39 2022] POST /identity/v3/users => generated 295 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:50:39.488562 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa498d4-a33a-4190-a922-fc27aead01ba admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:39.496729 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509367f-35d8-4d22-9c59-e5a4f07e750f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8f11d4419f946b9aa0accbc6a219488, user_id=e2a2f995befa475583e672a9e89ed281, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:39.498189 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa498d4-a33a-4190-a922-fc27aead01ba admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:39.499396 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9509367f-35d8-4d22-9c59-e5a4f07e750f admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:39.502947 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c9d10-3155-41a0-a822-f36a0ab4b900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:39.503541 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c9d10-3155-41a0-a822-f36a0ab4b900 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:39.503853 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c9d10-3155-41a0-a822-f36a0ab4b900 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:39.504127 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c9d10-3155-41a0-a822-f36a0ab4b900 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:39.504783 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c9d10-3155-41a0-a822-f36a0ab4b900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:39.506525 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c9d10-3155-41a0-a822-f36a0ab4b900 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:39.517609 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 630/1265] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:50:39.518557 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c9d10-3155-41a0-a822-f36a0ab4b900 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:39.528651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c9d10-3155-41a0-a822-f36a0ab4b900 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:39.533311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4109732-22b1-4802-a9db-eccdc521f398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:39.533938 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4109732-22b1-4802-a9db-eccdc521f398 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:39.534197 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4109732-22b1-4802-a9db-eccdc521f398 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:39.534464 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4109732-22b1-4802-a9db-eccdc521f398 admin admin] PATH_INFO: `/v3/projects/735cb7f647604741982528fc91abb7b6/users/6f1245ffc45b4c99990e746068bd852c/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:39.548059 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 636/1266] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:50:39 2022] PUT /identity/v3/projects/f8f11d4419f946b9aa0accbc6a219488/users/e2a2f995befa475583e672a9e89ed281/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:50:39.549213 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4109732-22b1-4802-a9db-eccdc521f398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=735cb7f647604741982528fc91abb7b6, user_id=6f1245ffc45b4c99990e746068bd852c, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:39.551988 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4109732-22b1-4802-a9db-eccdc521f398 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:39.558967 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509367f-35d8-4d22-9c59-e5a4f07e750f admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:39.592917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:39.607630 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 631/1267] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:50:39 2022] PUT /identity/v3/projects/735cb7f647604741982528fc91abb7b6/users/6f1245ffc45b4c99990e746068bd852c/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:50:39.609031 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4109732-22b1-4802-a9db-eccdc521f398 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:39.635666 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509367f-35d8-4d22-9c59-e5a4f07e750f admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:39.682814 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4109732-22b1-4802-a9db-eccdc521f398 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:39.687410 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b879a20d-6a94-4597-8c79-6670b5e84398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:39.688044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879a20d-6a94-4597-8c79-6670b5e84398 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:39.688317 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879a20d-6a94-4597-8c79-6670b5e84398 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:39.688585 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879a20d-6a94-4597-8c79-6670b5e84398 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:39.689217 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b879a20d-6a94-4597-8c79-6670b5e84398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:39.690860 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b879a20d-6a94-4597-8c79-6670b5e84398 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:39.694211 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f8cad-f7a0-4142-a384-b81405849965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:39.694877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f8cad-f7a0-4142-a384-b81405849965 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:39.695161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f8cad-f7a0-4142-a384-b81405849965 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:39.695437 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38f8cad-f7a0-4142-a384-b81405849965 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:39.696130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38f8cad-f7a0-4142-a384-b81405849965 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:39.697692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38f8cad-f7a0-4142-a384-b81405849965 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:39.702399 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 632/1268] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:50:39.703857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b879a20d-6a94-4597-8c79-6670b5e84398 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:39.707931 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 637/1269] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:50:39.708680 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f8cad-f7a0-4142-a384-b81405849965 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:39.715530 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b879a20d-6a94-4597-8c79-6670b5e84398 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:39.718273 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38f8cad-f7a0-4142-a384-b81405849965 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:39.720106 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82d1f4-3fd7-4e19-91f6-cdbda3d1473d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:39.720705 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82d1f4-3fd7-4e19-91f6-cdbda3d1473d admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:39.720988 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82d1f4-3fd7-4e19-91f6-cdbda3d1473d admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:39.721262 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82d1f4-3fd7-4e19-91f6-cdbda3d1473d admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:39.721896 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d82d1f4-3fd7-4e19-91f6-cdbda3d1473d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:39.723005 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d5a6b7-4b57-4e1d-81fa-39c172b73e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:39.723530 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d82d1f4-3fd7-4e19-91f6-cdbda3d1473d admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:39.723638 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d5a6b7-4b57-4e1d-81fa-39c172b73e44 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:39.723891 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d5a6b7-4b57-4e1d-81fa-39c172b73e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:39.724165 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d5a6b7-4b57-4e1d-81fa-39c172b73e44 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:39.724787 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d5a6b7-4b57-4e1d-81fa-39c172b73e44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:39.726872 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d5a6b7-4b57-4e1d-81fa-39c172b73e44 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:39.729348 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:39.735020 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 633/1270] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:50:39.736615 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82d1f4-3fd7-4e19-91f6-cdbda3d1473d admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:39.736739 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 638/1271] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:50:39.737407 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d5a6b7-4b57-4e1d-81fa-39c172b73e44 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:39.747247 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82d1f4-3fd7-4e19-91f6-cdbda3d1473d admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:39.747970 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d5a6b7-4b57-4e1d-81fa-39c172b73e44 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:39.751800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b67865-086f-4302-b622-575597dfbc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:39.752566 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b67865-086f-4302-b622-575597dfbc5b admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:39.752700 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34cad89-bbee-44d0-a298-7f5c6d7fc7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:39.753251 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34cad89-bbee-44d0-a298-7f5c6d7fc7d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:39.753376 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b67865-086f-4302-b622-575597dfbc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:39.753532 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34cad89-bbee-44d0-a298-7f5c6d7fc7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:39.753645 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b67865-086f-4302-b622-575597dfbc5b admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:39.753807 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34cad89-bbee-44d0-a298-7f5c6d7fc7d2 admin admin] PATH_INFO: `/v3/projects/38bd8dbff7d74bf4baa7c2aeca32c557/users/6307c4e28ee044dab595913514501537/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:39.754229 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b67865-086f-4302-b622-575597dfbc5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:39.756364 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b67865-086f-4302-b622-575597dfbc5b admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:39.767849 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 634/1272] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:50:39.767849 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022f154-c1ee-4efc-bcff-ee0f8ff13824 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:39.774551 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022f154-c1ee-4efc-bcff-ee0f8ff13824 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:39.774551 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022f154-c1ee-4efc-bcff-ee0f8ff13824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:39.774749 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34cad89-bbee-44d0-a298-7f5c6d7fc7d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38bd8dbff7d74bf4baa7c2aeca32c557, user_id=6307c4e28ee044dab595913514501537, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:39.774749 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34cad89-bbee-44d0-a298-7f5c6d7fc7d2 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:39.795417 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-6022f154-c1ee-4efc-bcff-ee0f8ff13824 None None] MFA Rules not processed for user `e2a2f995befa475583e672a9e89ed281`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:39.796697 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6022f154-c1ee-4efc-bcff-ee0f8ff13824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:39.838889 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 639/1273] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:50:39 2022] PUT /identity/v3/projects/38bd8dbff7d74bf4baa7c2aeca32c557/users/6307c4e28ee044dab595913514501537/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:50:39.840786 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437502f3-9ac1-486e-a72b-9312bf93a15a None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:39.841377 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437502f3-9ac1-486e-a72b-9312bf93a15a None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:39.841657 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437502f3-9ac1-486e-a72b-9312bf93a15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:39.860370 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 635/1274] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:39.860837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-437502f3-9ac1-486e-a72b-9312bf93a15a None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:39.861453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022f154-c1ee-4efc-bcff-ee0f8ff13824 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:39.862059 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-437502f3-9ac1-486e-a72b-9312bf93a15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:39.874504 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022f154-c1ee-4efc-bcff-ee0f8ff13824 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:39.934048 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 640/1275] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:39.934870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e224d-d6a2-441f-81c7-b6aee7eb14b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:39.935040 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437502f3-9ac1-486e-a72b-9312bf93a15a None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:39.935530 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e224d-d6a2-441f-81c7-b6aee7eb14b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:39.935841 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e224d-d6a2-441f-81c7-b6aee7eb14b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:39.936122 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e224d-d6a2-441f-81c7-b6aee7eb14b3 admin admin] PATH_INFO: `/v3/projects/cf238531f665437faddc8002d3b2ab6c/users/ec2ca63a9f4c4c18bde25b1e4eab93ab/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:39.944218 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437502f3-9ac1-486e-a72b-9312bf93a15a None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:39.948979 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8b117-03a5-455e-af33-30634b15f35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:39.949588 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8b117-03a5-455e-af33-30634b15f35f admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:39.949870 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8b117-03a5-455e-af33-30634b15f35f admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:39.950146 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe8b117-03a5-455e-af33-30634b15f35f admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:39.950785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe8b117-03a5-455e-af33-30634b15f35f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:39.952395 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e224d-d6a2-441f-81c7-b6aee7eb14b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf238531f665437faddc8002d3b2ab6c, user_id=ec2ca63a9f4c4c18bde25b1e4eab93ab, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:39.952538 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe8b117-03a5-455e-af33-30634b15f35f admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:39.955547 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e224d-d6a2-441f-81c7-b6aee7eb14b3 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:39.962938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 641/1276] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:50:39.963819 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8b117-03a5-455e-af33-30634b15f35f admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:39.973657 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe8b117-03a5-455e-af33-30634b15f35f admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:39.978753 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1fe35d-7b73-43de-9582-a03386315c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:39.979382 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fe35d-7b73-43de-9582-a03386315c69 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:39.979662 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fe35d-7b73-43de-9582-a03386315c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:39.979968 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fe35d-7b73-43de-9582-a03386315c69 admin admin] PATH_INFO: `/v3/projects/84929e7da65d4696913f3fc844f90854/users/7619ccd38dbe408bbde7f354dcb32111/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:39.996114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1fe35d-7b73-43de-9582-a03386315c69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84929e7da65d4696913f3fc844f90854, user_id=7619ccd38dbe408bbde7f354dcb32111, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:39.998961 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1fe35d-7b73-43de-9582-a03386315c69 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:40.022209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 636/1277] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:50:39 2022] PUT /identity/v3/projects/cf238531f665437faddc8002d3b2ab6c/users/ec2ca63a9f4c4c18bde25b1e4eab93ab/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:50:40.023826 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e224d-d6a2-441f-81c7-b6aee7eb14b3 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:40.083343 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e224d-d6a2-441f-81c7-b6aee7eb14b3 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:40.088683 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebc888-c20f-4d41-897b-fbb5ed07abe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:40.089414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebc888-c20f-4d41-897b-fbb5ed07abe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:40.089972 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebc888-c20f-4d41-897b-fbb5ed07abe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:40.090250 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebc888-c20f-4d41-897b-fbb5ed07abe5 admin admin] PATH_INFO: `/v3/projects/f9eae63ad8f14fc9a6432a5c846981de/users/15dac03b55094e93ac1d567c9c8f7bb3/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:40.094012 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:40.097321 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 642/1278] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:50:39 2022] PUT /identity/v3/projects/84929e7da65d4696913f3fc844f90854/users/7619ccd38dbe408bbde7f354dcb32111/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:50:40.098027 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1fe35d-7b73-43de-9582-a03386315c69 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:40.108105 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ebc888-c20f-4d41-897b-fbb5ed07abe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9eae63ad8f14fc9a6432a5c846981de, user_id=15dac03b55094e93ac1d567c9c8f7bb3, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:40.110801 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ebc888-c20f-4d41-897b-fbb5ed07abe5 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:40.115177 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:40.167860 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1fe35d-7b73-43de-9582-a03386315c69 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:40.171262 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 637/1279] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:50:40 2022] PUT /identity/v3/projects/f9eae63ad8f14fc9a6432a5c846981de/users/15dac03b55094e93ac1d567c9c8f7bb3/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:50:40.175795 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfcc38-5871-490a-a24e-e8ac10149930 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:40.175795 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfcc38-5871-490a-a24e-e8ac10149930 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:40.175795 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcfcc38-5871-490a-a24e-e8ac10149930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:40.193301 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-8fcfcc38-5871-490a-a24e-e8ac10149930 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:40.194533 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-8fcfcc38-5871-490a-a24e-e8ac10149930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:40.230504 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8f661-d75e-48d4-9285-487857aeedb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:40.231117 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8f661-d75e-48d4-9285-487857aeedb1 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:40.231399 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8f661-d75e-48d4-9285-487857aeedb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:40.231780 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8f661-d75e-48d4-9285-487857aeedb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:40.234195 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8f661-d75e-48d4-9285-487857aeedb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:40.234195 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8f661-d75e-48d4-9285-487857aeedb1 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:40.243766 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 643/1280] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:50:40 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:50:40.244939 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5a53b-e98e-4d69-aaf5-5b77c30991d6 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:40.245485 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5a53b-e98e-4d69-aaf5-5b77c30991d6 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:40.245485 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5a53b-e98e-4d69-aaf5-5b77c30991d6 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:40.247873 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 644/1281] 10.222.0.17 () {54 vars in 961 bytes} [Mon Oct 17 03:50:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:50:40.249277 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b5a53b-e98e-4d69-aaf5-5b77c30991d6 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:40.255095 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 638/1282] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:40.256001 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfcc38-5871-490a-a24e-e8ac10149930 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:40.265847 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcfcc38-5871-490a-a24e-e8ac10149930 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:40.270924 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4f59c-b4a8-4d06-823a-21822fd2ba7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:40.277222 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4f59c-b4a8-4d06-823a-21822fd2ba7f admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:40.277222 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4f59c-b4a8-4d06-823a-21822fd2ba7f admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:40.277222 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4f59c-b4a8-4d06-823a-21822fd2ba7f admin admin] PATH_INFO: `/v3/projects/3a5c79ca08614d7ba9cad0da6f32cccf/users/5bcf4a4b0ab14a118be938a5ee980e08/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:40.287891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4f59c-b4a8-4d06-823a-21822fd2ba7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a5c79ca08614d7ba9cad0da6f32cccf, user_id=5bcf4a4b0ab14a118be938a5ee980e08, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:40.291123 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a4f59c-b4a8-4d06-823a-21822fd2ba7f admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:40.306483 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b5a53b-e98e-4d69-aaf5-5b77c30991d6 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:40.310189 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a02c9f-7a92-406a-90c6-b1f0aa4b8066 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:40.310791 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a02c9f-7a92-406a-90c6-b1f0aa4b8066 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:40.311072 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a02c9f-7a92-406a-90c6-b1f0aa4b8066 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:40.311342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a02c9f-7a92-406a-90c6-b1f0aa4b8066 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:40.320452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a02c9f-7a92-406a-90c6-b1f0aa4b8066 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:40.321873 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a02c9f-7a92-406a-90c6-b1f0aa4b8066 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:40.331074 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 645/1283] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:50:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:50:40.340609 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e0c03-bdc9-4be0-87f8-569b371a0b30 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:40.340609 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e0c03-bdc9-4be0-87f8-569b371a0b30 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:40.340609 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e0c03-bdc9-4be0-87f8-569b371a0b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:40.351986 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-af9e0c03-bdc9-4be0-87f8-569b371a0b30 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:40.353189 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-af9e0c03-bdc9-4be0-87f8-569b371a0b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:40.360861 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 646/1284] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:40.362156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c513782-b361-48e4-8cfb-eef5ecef218c None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:40.362470 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c513782-b361-48e4-8cfb-eef5ecef218c None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:40.362747 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c513782-b361-48e4-8cfb-eef5ecef218c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:40.378158 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 639/1285] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:50:40 2022] PUT /identity/v3/projects/3a5c79ca08614d7ba9cad0da6f32cccf/users/5bcf4a4b0ab14a118be938a5ee980e08/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:50:40.378158 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb5e4b-61e5-41ee-9067-9e7f6f1106c2 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:40.378158 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb5e4b-61e5-41ee-9067-9e7f6f1106c2 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:40.378158 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb5e4b-61e5-41ee-9067-9e7f6f1106c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:40.391826 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-0c513782-b361-48e4-8cfb-eef5ecef218c None None] MFA Rules not processed for user `7619ccd38dbe408bbde7f354dcb32111`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:40.393078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-0c513782-b361-48e4-8cfb-eef5ecef218c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:40.405206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-d4bb5e4b-61e5-41ee-9067-9e7f6f1106c2 None None] MFA Rules not processed for user `15dac03b55094e93ac1d567c9c8f7bb3`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:40.406378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-d4bb5e4b-61e5-41ee-9067-9e7f6f1106c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:40.434216 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:40.467453 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 647/1286] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:40.468326 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c513782-b361-48e4-8cfb-eef5ecef218c None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:40.494961 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 640/1287] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:40.496274 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39de2bb-d3cf-448d-9fc6-7bb39cc0eb62 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:40.496559 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39de2bb-d3cf-448d-9fc6-7bb39cc0eb62 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:40.496846 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39de2bb-d3cf-448d-9fc6-7bb39cc0eb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:40.544404 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c513782-b361-48e4-8cfb-eef5ecef218c None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:40.548962 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ff3f6-7854-4d4e-a448-40b0b6a2c470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:40.549620 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ff3f6-7854-4d4e-a448-40b0b6a2c470 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:40.549727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-c39de2bb-d3cf-448d-9fc6-7bb39cc0eb62 None None] MFA Rules not processed for user `d961b751763a4e49b5d4ca4c795c5bf6`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:40.549899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ff3f6-7854-4d4e-a448-40b0b6a2c470 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:40.550170 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926ff3f6-7854-4d4e-a448-40b0b6a2c470 admin admin] PATH_INFO: `/v3/projects/3ed4ecab9c234dd293ba0034d71d03f7/users/57ee3a13e33549c9a90ebde353fcd6f2/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:40.550834 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-c39de2bb-d3cf-448d-9fc6-7bb39cc0eb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:40.565296 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ff3f6-7854-4d4e-a448-40b0b6a2c470 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ed4ecab9c234dd293ba0034d71d03f7, user_id=57ee3a13e33549c9a90ebde353fcd6f2, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:40.568117 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926ff3f6-7854-4d4e-a448-40b0b6a2c470 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:40.584923 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:40.619215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 641/1288] 10.222.0.17 () {58 vars in 1058 bytes} [Mon Oct 17 03:50:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:40.620009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39de2bb-d3cf-448d-9fc6-7bb39cc0eb62 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:40.621969 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 648/1289] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:50:40 2022] PUT /identity/v3/projects/3ed4ecab9c234dd293ba0034d71d03f7/users/57ee3a13e33549c9a90ebde353fcd6f2/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:50:40.622777 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ff3f6-7854-4d4e-a448-40b0b6a2c470 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:40.696823 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926ff3f6-7854-4d4e-a448-40b0b6a2c470 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:40.697434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39de2bb-d3cf-448d-9fc6-7bb39cc0eb62 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:40.702394 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c306ce-8a36-4827-a059-f397a604059d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:40.703159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c306ce-8a36-4827-a059-f397a604059d service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:40.703580 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c306ce-8a36-4827-a059-f397a604059d service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:40.703955 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c306ce-8a36-4827-a059-f397a604059d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:40.704826 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9bc5d9-ea57-4bbf-aa3c-3916d44f735f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:40.705626 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9bc5d9-ea57-4bbf-aa3c-3916d44f735f service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:40.705979 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9bc5d9-ea57-4bbf-aa3c-3916d44f735f service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:40.706225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9bc5d9-ea57-4bbf-aa3c-3916d44f735f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:40.765543 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c306ce-8a36-4827-a059-f397a604059d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:40.767021 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c306ce-8a36-4827-a059-f397a604059d service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:40.767659 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9bc5d9-ea57-4bbf-aa3c-3916d44f735f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:40.769139 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9bc5d9-ea57-4bbf-aa3c-3916d44f735f service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:40.778698 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 642/1290] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:50:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:50:40.779154 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 649/1291] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:50:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:50:40.780192 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a750a-2916-477a-93e3-cddbe2f5182b None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:40.780452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639821c-b85e-43d8-add0-7e99df464a62 None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:40.780739 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639821c-b85e-43d8-add0-7e99df464a62 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:40.781013 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639821c-b85e-43d8-add0-7e99df464a62 None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:40.781327 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a750a-2916-477a-93e3-cddbe2f5182b None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:40.781674 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a750a-2916-477a-93e3-cddbe2f5182b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:40.782372 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 650/1292] 10.222.0.17 () {54 vars in 961 bytes} [Mon Oct 17 03:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:50:40.782981 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639821c-b85e-43d8-add0-7e99df464a62 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:40.793914 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:40.813727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-727a750a-2916-477a-93e3-cddbe2f5182b None None] MFA Rules not processed for user `5bcf4a4b0ab14a118be938a5ee980e08`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:40.815166 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-727a750a-2916-477a-93e3-cddbe2f5182b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:40.843660 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639821c-b85e-43d8-add0-7e99df464a62 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:40.847909 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f429229-5d34-4541-9234-03d98bdf83b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:40.848377 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f429229-5d34-4541-9234-03d98bdf83b3 service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:40.848661 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f429229-5d34-4541-9234-03d98bdf83b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:40.848935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f429229-5d34-4541-9234-03d98bdf83b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:40.886754 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 643/1293] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:40 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:40.888303 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b789cac-3b93-4850-8b5b-9462b65143bb None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:40.888659 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b789cac-3b93-4850-8b5b-9462b65143bb None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:40.889386 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b789cac-3b93-4850-8b5b-9462b65143bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:40.912654 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-2b789cac-3b93-4850-8b5b-9462b65143bb None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:40.913907 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-2b789cac-3b93-4850-8b5b-9462b65143bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:40.914312 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f429229-5d34-4541-9234-03d98bdf83b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:40.915867 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f429229-5d34-4541-9234-03d98bdf83b3 service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:40.925500 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 651/1294] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:50:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:50:40.926892 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f429229-5d34-4541-9234-03d98bdf83b3 service cinder] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:40.929619 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 644/1295] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:40.933072 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97985c52-16fd-4b6c-99f3-702a63a30640 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:40.933423 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97985c52-16fd-4b6c-99f3-702a63a30640 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:40.933662 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97985c52-16fd-4b6c-99f3-702a63a30640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:40.945463 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:40.968504 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-97985c52-16fd-4b6c-99f3-702a63a30640 None None] MFA Rules not processed for user `d961b751763a4e49b5d4ca4c795c5bf6`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:40.969846 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-97985c52-16fd-4b6c-99f3-702a63a30640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:41.053511 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f429229-5d34-4541-9234-03d98bdf83b3 service cinder] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:41.076986 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b6ed5-8950-43b9-9553-e08ebfc6706b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:41.078154 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b6ed5-8950-43b9-9553-e08ebfc6706b service nova] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:41.078499 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b6ed5-8950-43b9-9553-e08ebfc6706b service nova] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:41.078905 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6b6ed5-8950-43b9-9553-e08ebfc6706b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:41.126245 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 645/1296] 10.222.0.17 () {58 vars in 1058 bytes} [Mon Oct 17 03:50:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:41.127354 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97985c52-16fd-4b6c-99f3-702a63a30640 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:41.155185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97985c52-16fd-4b6c-99f3-702a63a30640 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:41.168283 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b6ed5-8950-43b9-9553-e08ebfc6706b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:41.169985 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6b6ed5-8950-43b9-9553-e08ebfc6706b service nova] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:41.170169 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e93308-a0d1-4407-9a13-148afbc8b948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:41.170812 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e93308-a0d1-4407-9a13-148afbc8b948 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:41.171095 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e93308-a0d1-4407-9a13-148afbc8b948 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:41.171370 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e93308-a0d1-4407-9a13-148afbc8b948 admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:41.172661 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e93308-a0d1-4407-9a13-148afbc8b948 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:41.174738 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e93308-a0d1-4407-9a13-148afbc8b948 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:41.177978 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:41.198504 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 652/1297] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:50:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:50:41.200085 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b6ed5-8950-43b9-9553-e08ebfc6706b service nova] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:41.207856 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-ba6b6ed5-8950-43b9-9553-e08ebfc6706b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:41.228111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6b6ed5-8950-43b9-9553-e08ebfc6706b service nova] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:41.233427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ec0db-4b9c-405b-9e3e-9c5f6bd1be00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:41.234288 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ec0db-4b9c-405b-9e3e-9c5f6bd1be00 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:41.234956 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ec0db-4b9c-405b-9e3e-9c5f6bd1be00 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:41.235509 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ec0db-4b9c-405b-9e3e-9c5f6bd1be00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:41.247471 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ec0db-4b9c-405b-9e3e-9c5f6bd1be00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:41.249914 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ec0db-4b9c-405b-9e3e-9c5f6bd1be00 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:41.261773 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 653/1298] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:50:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:50:41.282642 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ec0db-4b9c-405b-9e3e-9c5f6bd1be00 service neutron] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:41.299004 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 646/1299] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:50:41 2022] POST /identity/v3/projects => generated 355 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:50:41.299975 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e93308-a0d1-4407-9a13-148afbc8b948 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:41.325141 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ec0db-4b9c-405b-9e3e-9c5f6bd1be00 service neutron] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:41.445061 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e93308-a0d1-4407-9a13-148afbc8b948 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:41.452241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83850fa-d561-4324-9f7f-cfacc3e07c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:41.455682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83850fa-d561-4324-9f7f-cfacc3e07c71 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:41.455682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83850fa-d561-4324-9f7f-cfacc3e07c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:41.455682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83850fa-d561-4324-9f7f-cfacc3e07c71 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:41.455682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83850fa-d561-4324-9f7f-cfacc3e07c71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:41.457761 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b4f2b-592f-4ed0-aec4-441d2568eff1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:41.457761 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b4f2b-592f-4ed0-aec4-441d2568eff1 service nova] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:41.457761 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b4f2b-592f-4ed0-aec4-441d2568eff1 service nova] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:41.458167 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83850fa-d561-4324-9f7f-cfacc3e07c71 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:41.458267 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b4f2b-592f-4ed0-aec4-441d2568eff1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:41.551287 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 647/1300] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:50:41 2022] POST /identity/v3/users => generated 284 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:50:41.553387 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40009c1b-c8bd-4479-bf9b-8be680b2d06d None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:41.555318 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40009c1b-c8bd-4479-bf9b-8be680b2d06d None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:41.555639 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40009c1b-c8bd-4479-bf9b-8be680b2d06d None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:41.557270 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 648/1301] 10.222.0.17 () {54 vars in 912 bytes} [Mon Oct 17 03:50:41 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:50:41.559986 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40009c1b-c8bd-4479-bf9b-8be680b2d06d None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:41.604458 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40009c1b-c8bd-4479-bf9b-8be680b2d06d None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:41.615697 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17235d4c-8c6e-40c4-9746-4121e546e93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:41.616376 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17235d4c-8c6e-40c4-9746-4121e546e93d admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:41.616733 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17235d4c-8c6e-40c4-9746-4121e546e93d admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:41.617028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17235d4c-8c6e-40c4-9746-4121e546e93d admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:41.618354 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17235d4c-8c6e-40c4-9746-4121e546e93d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:41.620108 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17235d4c-8c6e-40c4-9746-4121e546e93d admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:41.628385 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b4f2b-592f-4ed0-aec4-441d2568eff1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:41.630523 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b4f2b-592f-4ed0-aec4-441d2568eff1 service nova] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:41.636695 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 649/1302] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:50:41.642024 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13178d59-7ac2-4abc-8fa6-3dc49c43525f None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:41.642362 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13178d59-7ac2-4abc-8fa6-3dc49c43525f None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:41.642632 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13178d59-7ac2-4abc-8fa6-3dc49c43525f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:41.646991 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 654/1303] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:50:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:50:41.648420 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b4f2b-592f-4ed0-aec4-441d2568eff1 service nova] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:41.660636 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b4f2b-592f-4ed0-aec4-441d2568eff1 service nova] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:41.672127 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f7d6f-f81a-4137-a434-c01db92e7832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:41.675252 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f7d6f-f81a-4137-a434-c01db92e7832 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:41.675550 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f7d6f-f81a-4137-a434-c01db92e7832 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:41.675878 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f7d6f-f81a-4137-a434-c01db92e7832 admin admin] PATH_INFO: `/v3/projects/ef1cc4bcedbe45a3af49017162634616/users/3f9e4c1a43084fe58666277e03958238/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:41.692841 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-13178d59-7ac2-4abc-8fa6-3dc49c43525f None None] MFA Rules not processed for user `d961b751763a4e49b5d4ca4c795c5bf6`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:41.694134 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-13178d59-7ac2-4abc-8fa6-3dc49c43525f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:41.712067 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f7d6f-f81a-4137-a434-c01db92e7832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef1cc4bcedbe45a3af49017162634616, user_id=3f9e4c1a43084fe58666277e03958238, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:41.713818 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f7d6f-f81a-4137-a434-c01db92e7832 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:41.720035 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 650/1304] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Oct 17 03:50:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:41.734519 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13178d59-7ac2-4abc-8fa6-3dc49c43525f None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:41.802955 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:41.810708 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 655/1305] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:50:41 2022] PUT /identity/v3/projects/ef1cc4bcedbe45a3af49017162634616/users/3f9e4c1a43084fe58666277e03958238/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:50:41.812178 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f7d6f-f81a-4137-a434-c01db92e7832 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:41.820071 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13178d59-7ac2-4abc-8fa6-3dc49c43525f None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:41.928052 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211f456-c4ff-48d6-9e61-89a3f92bfc2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:41.928715 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211f456-c4ff-48d6-9e61-89a3f92bfc2f service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:41.928988 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211f456-c4ff-48d6-9e61-89a3f92bfc2f service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:41.929257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211f456-c4ff-48d6-9e61-89a3f92bfc2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:41.965034 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f7d6f-f81a-4137-a434-c01db92e7832 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:41.971613 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88c2fa-78f7-40da-988e-1424b81fd249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:41.972305 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88c2fa-78f7-40da-988e-1424b81fd249 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:41.972643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88c2fa-78f7-40da-988e-1424b81fd249 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:41.972925 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88c2fa-78f7-40da-988e-1424b81fd249 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:41.973664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba88c2fa-78f7-40da-988e-1424b81fd249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:41.975825 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba88c2fa-78f7-40da-988e-1424b81fd249 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:41.977479 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:41.996090 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 656/1306] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:50:41 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:50:41.997069 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88c2fa-78f7-40da-988e-1424b81fd249 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:42.025567 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88c2fa-78f7-40da-988e-1424b81fd249 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:42.038328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2c646-7952-4258-9338-6a3447fc54b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:42.040217 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2c646-7952-4258-9338-6a3447fc54b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:42.040527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2c646-7952-4258-9338-6a3447fc54b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:42.041491 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2c646-7952-4258-9338-6a3447fc54b9 admin admin] PATH_INFO: `/v3/projects/ef1cc4bcedbe45a3af49017162634616/users/3f9e4c1a43084fe58666277e03958238/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:42.047040 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd2c646-7952-4258-9338-6a3447fc54b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef1cc4bcedbe45a3af49017162634616, user_id=3f9e4c1a43084fe58666277e03958238, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:42.051291 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd2c646-7952-4258-9338-6a3447fc54b9 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:42.084663 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211f456-c4ff-48d6-9e61-89a3f92bfc2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:42.095328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 657/1307] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:50:41 2022] PUT /identity/v3/projects/ef1cc4bcedbe45a3af49017162634616/users/3f9e4c1a43084fe58666277e03958238/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:50:42.097062 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211f456-c4ff-48d6-9e61-89a3f92bfc2f service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:42.108449 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0d133-5aa3-41ed-ab4a-7502caee42f9 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:42.108449 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0d133-5aa3-41ed-ab4a-7502caee42f9 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:42.108449 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0d133-5aa3-41ed-ab4a-7502caee42f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:42.134152 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-e2c0d133-5aa3-41ed-ab4a-7502caee42f9 None None] MFA Rules not processed for user `3f9e4c1a43084fe58666277e03958238`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:42.135531 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-e2c0d133-5aa3-41ed-ab4a-7502caee42f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:42.138954 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 651/1308] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:50:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:50:42.142260 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb02206-ddef-4775-9f2f-270eddff8eba None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:42.142983 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb02206-ddef-4775-9f2f-270eddff8eba None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:42.143320 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb02206-ddef-4775-9f2f-270eddff8eba None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:42.148355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 652/1309] 10.222.0.17 () {54 vars in 912 bytes} [Mon Oct 17 03:50:42 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:50:42.158872 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5357d-dacc-459b-b10c-c0c147c5f29e None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:42.159157 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5357d-dacc-459b-b10c-c0c147c5f29e None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:42.168439 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d5357d-dacc-459b-b10c-c0c147c5f29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:42.217907 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-09d5357d-dacc-459b-b10c-c0c147c5f29e None None] MFA Rules not processed for user `d961b751763a4e49b5d4ca4c795c5bf6`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:42.221458 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-09d5357d-dacc-459b-b10c-c0c147c5f29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:42.286291 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 658/1310] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:42 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:42.301445 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0d133-5aa3-41ed-ab4a-7502caee42f9 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:42.353091 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 653/1311] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Oct 17 03:50:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:42.417437 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0d133-5aa3-41ed-ab4a-7502caee42f9 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:42.425760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60981a16-b521-467d-8bf7-89c33b4c2eca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:42.429898 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60981a16-b521-467d-8bf7-89c33b4c2eca service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:42.430230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60981a16-b521-467d-8bf7-89c33b4c2eca service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:42.430768 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60981a16-b521-467d-8bf7-89c33b4c2eca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:42.453977 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60981a16-b521-467d-8bf7-89c33b4c2eca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:42.465078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60981a16-b521-467d-8bf7-89c33b4c2eca service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:42.480036 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 659/1312] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:50:42 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:50:42.637942 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5357d-dacc-459b-b10c-c0c147c5f29e None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:42.648951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d5357d-dacc-459b-b10c-c0c147c5f29e None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:42.656031 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eddc1e-3508-45d2-9af7-cf520727292f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:42.656964 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eddc1e-3508-45d2-9af7-cf520727292f service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:42.657444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eddc1e-3508-45d2-9af7-cf520727292f service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:42.657871 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eddc1e-3508-45d2-9af7-cf520727292f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:42.681356 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eddc1e-3508-45d2-9af7-cf520727292f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:42.686233 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eddc1e-3508-45d2-9af7-cf520727292f service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:42.719785 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 654/1313] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:50:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:50:43.714572 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60981a16-b521-467d-8bf7-89c33b4c2eca service cinder] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:43.734722 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60981a16-b521-467d-8bf7-89c33b4c2eca service cinder] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:43.743178 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663488ae-2d9c-440e-a27a-9aca2ef7b65d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:43.744324 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663488ae-2d9c-440e-a27a-9aca2ef7b65d service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:43.744822 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663488ae-2d9c-440e-a27a-9aca2ef7b65d service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:43.745641 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663488ae-2d9c-440e-a27a-9aca2ef7b65d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:43.961490 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663488ae-2d9c-440e-a27a-9aca2ef7b65d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:43.963008 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663488ae-2d9c-440e-a27a-9aca2ef7b65d service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:43.994410 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 660/1314] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:50:43 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:50:46.547989 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eddc1e-3508-45d2-9af7-cf520727292f service cinder] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:46.602766 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:46.713878 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eddc1e-3508-45d2-9af7-cf520727292f service cinder] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:46.719531 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4fa548-43d9-435e-932a-60640f0d01c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:46.720292 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fa548-43d9-435e-932a-60640f0d01c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:46.720582 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fa548-43d9-435e-932a-60640f0d01c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:46.720867 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4fa548-43d9-435e-932a-60640f0d01c2 admin admin] PATH_INFO: `/v3/users/e2a2f995befa475583e672a9e89ed281` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:46.723495 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4fa548-43d9-435e-932a-60640f0d01c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2a2f995befa475583e672a9e89ed281)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:46.737553 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4fa548-43d9-435e-932a-60640f0d01c2 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:46.938999 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-0d4fa548-43d9-435e-932a-60640f0d01c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2a2f995befa475583e672a9e89ed281'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:46.940375 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-0d4fa548-43d9-435e-932a-60640f0d01c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2a2f995befa475583e672a9e89ed281'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:47.028195 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-0d4fa548-43d9-435e-932a-60640f0d01c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2a2f995befa475583e672a9e89ed281'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:47.074169 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-0d4fa548-43d9-435e-932a-60640f0d01c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2a2f995befa475583e672a9e89ed281'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:47.078509 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-0d4fa548-43d9-435e-932a-60640f0d01c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2a2f995befa475583e672a9e89ed281'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:47.093360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 655/1315] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:50:46 2022] DELETE /identity/v3/users/e2a2f995befa475583e672a9e89ed281 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:50:47.095064 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73763517-4b28-4189-b01b-3e7b379beafd None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:47.095561 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73763517-4b28-4189-b01b-3e7b379beafd None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:47.095685 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73763517-4b28-4189-b01b-3e7b379beafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:47.180720 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-73763517-4b28-4189-b01b-3e7b379beafd None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:47.182272 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-73763517-4b28-4189-b01b-3e7b379beafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:47.198506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:47.254921 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 661/1316] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:47.266649 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4fa548-43d9-435e-932a-60640f0d01c2 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:47.268379 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-0d4fa548-43d9-435e-932a-60640f0d01c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:47.328314 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4fa548-43d9-435e-932a-60640f0d01c2 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:47.332533 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488a703-a8a7-44dc-a72a-dc00240cea79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:47.333145 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488a703-a8a7-44dc-a72a-dc00240cea79 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:47.333808 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488a703-a8a7-44dc-a72a-dc00240cea79 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:47.333808 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488a703-a8a7-44dc-a72a-dc00240cea79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:47.343747 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e488a703-a8a7-44dc-a72a-dc00240cea79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:47.345195 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e488a703-a8a7-44dc-a72a-dc00240cea79 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:47.355513 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 656/1317] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:50:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:50:47.941304 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73763517-4b28-4189-b01b-3e7b379beafd None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:47.941723 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-73763517-4b28-4189-b01b-3e7b379beafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:47.964733 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73763517-4b28-4189-b01b-3e7b379beafd None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:47.981029 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc38f48-e779-49ff-90fb-883456740d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:47.981916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc38f48-e779-49ff-90fb-883456740d80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:47.982660 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc38f48-e779-49ff-90fb-883456740d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:47.983300 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc38f48-e779-49ff-90fb-883456740d80 admin admin] PATH_INFO: `/v3/projects/f8f11d4419f946b9aa0accbc6a219488` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:47.984945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc38f48-e779-49ff-90fb-883456740d80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8f11d4419f946b9aa0accbc6a219488)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:47.987537 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc38f48-e779-49ff-90fb-883456740d80 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:48.248352 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-5dc38f48-e779-49ff-90fb-883456740d80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8f11d4419f946b9aa0accbc6a219488', 'request_id': 'req-5dc38f48-e779-49ff-90fb-883456740d80'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:48.325420 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-5dc38f48-e779-49ff-90fb-883456740d80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8f11d4419f946b9aa0accbc6a219488', 'request_id': 'req-5dc38f48-e779-49ff-90fb-883456740d80'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:48.333126 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-5dc38f48-e779-49ff-90fb-883456740d80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8f11d4419f946b9aa0accbc6a219488', 'request_id': 'req-5dc38f48-e779-49ff-90fb-883456740d80'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:48.378978 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-5dc38f48-e779-49ff-90fb-883456740d80 admin admin] The token cache is being invalidate because project f8f11d4419f946b9aa0accbc6a219488 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:50:48.379717 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-5dc38f48-e779-49ff-90fb-883456740d80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:48.380310 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-5dc38f48-e779-49ff-90fb-883456740d80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:48.386063 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 662/1318] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:50:47 2022] DELETE /identity/v3/projects/f8f11d4419f946b9aa0accbc6a219488 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:50:48.388307 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace454a-ada3-4c32-992d-a5c6d93ab83d None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:48.389283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace454a-ada3-4c32-992d-a5c6d93ab83d None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:48.389807 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace454a-ada3-4c32-992d-a5c6d93ab83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:48.430243 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-6ace454a-ada3-4c32-992d-a5c6d93ab83d None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:48.431967 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6ace454a-ada3-4c32-992d-a5c6d93ab83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:48.506185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:48.535385 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 657/1319] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:48.544073 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d457fb6-738c-4c0e-9ae4-f1066d3713b2 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:48.544605 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d457fb6-738c-4c0e-9ae4-f1066d3713b2 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:48.545052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d457fb6-738c-4c0e-9ae4-f1066d3713b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:48.573471 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-7d457fb6-738c-4c0e-9ae4-f1066d3713b2 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:48.575346 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-7d457fb6-738c-4c0e-9ae4-f1066d3713b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:48.587762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 663/1320] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:48.590854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace454a-ada3-4c32-992d-a5c6d93ab83d None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:48.606337 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace454a-ada3-4c32-992d-a5c6d93ab83d None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:48.620600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0c1b9-17fe-43ab-aba1-fddab61fea0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:48.621640 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0c1b9-17fe-43ab-aba1-fddab61fea0d admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:48.622127 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0c1b9-17fe-43ab-aba1-fddab61fea0d admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:48.622590 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0c1b9-17fe-43ab-aba1-fddab61fea0d admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:48.623451 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0c1b9-17fe-43ab-aba1-fddab61fea0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:50:48.624867 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0c1b9-17fe-43ab-aba1-fddab61fea0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:48.632520 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0c1b9-17fe-43ab-aba1-fddab61fea0d admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:48.666757 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 658/1321] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:50:48.673906 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d457fb6-738c-4c0e-9ae4-f1066d3713b2 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:48.689933 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d457fb6-738c-4c0e-9ae4-f1066d3713b2 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:48.696614 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25433758-f165-47f3-b9e8-e4e534fd991b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:48.699304 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25433758-f165-47f3-b9e8-e4e534fd991b admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:48.703640 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25433758-f165-47f3-b9e8-e4e534fd991b admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:48.704156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25433758-f165-47f3-b9e8-e4e534fd991b admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:48.704949 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25433758-f165-47f3-b9e8-e4e534fd991b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:48.709941 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25433758-f165-47f3-b9e8-e4e534fd991b admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:48.880481 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0c1b9-17fe-43ab-aba1-fddab61fea0d admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:48.881160 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 664/1322] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:50:48 2022] POST /identity/v3/projects => generated 373 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:50:48.893244 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0c1b9-17fe-43ab-aba1-fddab61fea0d admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:49.055541 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc24ae-4ff4-483f-b9ce-1e2ad6692688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:49.056723 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc24ae-4ff4-483f-b9ce-1e2ad6692688 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:49.057008 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc24ae-4ff4-483f-b9ce-1e2ad6692688 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:49.058443 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc24ae-4ff4-483f-b9ce-1e2ad6692688 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:49.064420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc24ae-4ff4-483f-b9ce-1e2ad6692688 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:49.067179 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc24ae-4ff4-483f-b9ce-1e2ad6692688 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:49.249344 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 659/1323] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:50:48 2022] POST /identity/v3/users => generated 287 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:50:49.253187 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25433758-f165-47f3-b9e8-e4e534fd991b admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:49.271268 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25433758-f165-47f3-b9e8-e4e534fd991b admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:49.273057 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbedf5d-4b3b-4d69-b776-46b19de4797a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:49.273882 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbedf5d-4b3b-4d69-b776-46b19de4797a admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:49.274371 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbedf5d-4b3b-4d69-b776-46b19de4797a admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:49.274803 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbedf5d-4b3b-4d69-b776-46b19de4797a admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:49.275615 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbedf5d-4b3b-4d69-b776-46b19de4797a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:49.277666 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbedf5d-4b3b-4d69-b776-46b19de4797a admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:49.291007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 665/1324] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:50:49.294710 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc24ae-4ff4-483f-b9ce-1e2ad6692688 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:49.308482 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc24ae-4ff4-483f-b9ce-1e2ad6692688 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:49.316233 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af3dac-18d5-4c4e-914d-d262f6000cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:49.317592 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af3dac-18d5-4c4e-914d-d262f6000cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:49.317970 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af3dac-18d5-4c4e-914d-d262f6000cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:49.318388 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af3dac-18d5-4c4e-914d-d262f6000cd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:49.319822 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af3dac-18d5-4c4e-914d-d262f6000cd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:49.321918 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af3dac-18d5-4c4e-914d-d262f6000cd0 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:49.357252 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 660/1325] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:50:49.360753 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbedf5d-4b3b-4d69-b776-46b19de4797a admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:49.393011 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbedf5d-4b3b-4d69-b776-46b19de4797a admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:49.415338 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4dda03-546f-4e53-9cba-7f928c176c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:49.416379 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4dda03-546f-4e53-9cba-7f928c176c1f admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:49.416858 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4dda03-546f-4e53-9cba-7f928c176c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:49.417528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4dda03-546f-4e53-9cba-7f928c176c1f admin admin] PATH_INFO: `/v3/projects/af361498dd7b4973b66b005b255b3828/users/4e5cd48dc3b4421594da56091edf3367/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:49.469302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4dda03-546f-4e53-9cba-7f928c176c1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=af361498dd7b4973b66b005b255b3828, user_id=4e5cd48dc3b4421594da56091edf3367, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:49.472044 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4dda03-546f-4e53-9cba-7f928c176c1f admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:49.563047 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 666/1326] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:50:49 2022] PUT /identity/v3/projects/af361498dd7b4973b66b005b255b3828/users/4e5cd48dc3b4421594da56091edf3367/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:50:49.576126 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911cfdc-a191-492f-abbd-268acb233e4e None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:49.579226 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911cfdc-a191-492f-abbd-268acb233e4e None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:49.579226 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911cfdc-a191-492f-abbd-268acb233e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:49.635683 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-d911cfdc-a191-492f-abbd-268acb233e4e None None] MFA Rules not processed for user `4e5cd48dc3b4421594da56091edf3367`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:49.637236 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-d911cfdc-a191-492f-abbd-268acb233e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:49.726905 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 661/1327] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:49.731810 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4dda03-546f-4e53-9cba-7f928c176c1f admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:49.801894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:49.874770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4dda03-546f-4e53-9cba-7f928c176c1f admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:49.889826 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ec4f8-0c9b-4815-87ae-f9954e1afbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:49.895205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ec4f8-0c9b-4815-87ae-f9954e1afbf6 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:49.895205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ec4f8-0c9b-4815-87ae-f9954e1afbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:49.895205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ec4f8-0c9b-4815-87ae-f9954e1afbf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:49.895205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ec4f8-0c9b-4815-87ae-f9954e1afbf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:49.899823 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ec4f8-0c9b-4815-87ae-f9954e1afbf6 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:50.058014 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 667/1328] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:50:49 2022] POST /identity/v3/projects => generated 373 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:50:50.059106 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911cfdc-a191-492f-abbd-268acb233e4e None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:50.076976 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d911cfdc-a191-492f-abbd-268acb233e4e None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:50.126411 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:50.220588 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8cb86-e633-4045-bf33-f8463adba8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:50.221355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8cb86-e633-4045-bf33-f8463adba8a0 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:50.221733 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8cb86-e633-4045-bf33-f8463adba8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:50.222147 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8cb86-e633-4045-bf33-f8463adba8a0 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:50.223257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8cb86-e633-4045-bf33-f8463adba8a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:50.227236 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8cb86-e633-4045-bf33-f8463adba8a0 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:50.347802 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 662/1329] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:50:50 2022] POST /identity/v3/users => generated 293 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:50:50.349654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ec4f8-0c9b-4815-87ae-f9954e1afbf6 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:50.363379 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ec4f8-0c9b-4815-87ae-f9954e1afbf6 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:50.371448 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4f12e-0e81-42a2-aa94-dc416c1d9174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:50.373870 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4f12e-0e81-42a2-aa94-dc416c1d9174 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:50.374383 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4f12e-0e81-42a2-aa94-dc416c1d9174 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:50.374988 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a4f12e-0e81-42a2-aa94-dc416c1d9174 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:50.380120 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a4f12e-0e81-42a2-aa94-dc416c1d9174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:50.384032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a4f12e-0e81-42a2-aa94-dc416c1d9174 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:50.414688 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8cb86-e633-4045-bf33-f8463adba8a0 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:50.424844 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 668/1330] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:50:50.430739 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8cb86-e633-4045-bf33-f8463adba8a0 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:50.441024 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a719b9-f5a5-4984-b9f3-7bebcf08bd1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:50.441024 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a719b9-f5a5-4984-b9f3-7bebcf08bd1b admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:50.441591 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a719b9-f5a5-4984-b9f3-7bebcf08bd1b admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:50.441732 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a719b9-f5a5-4984-b9f3-7bebcf08bd1b admin admin] PATH_INFO: `/v3/projects/b1ab77d2ff914022bbdf3b83eb442d57/users/af0baf4ed1c04420921c9d87aed78e85/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:50.466031 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a719b9-f5a5-4984-b9f3-7bebcf08bd1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1ab77d2ff914022bbdf3b83eb442d57, user_id=af0baf4ed1c04420921c9d87aed78e85, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:50.470321 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a719b9-f5a5-4984-b9f3-7bebcf08bd1b admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:50.603306 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 663/1331] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:50:50 2022] PUT /identity/v3/projects/b1ab77d2ff914022bbdf3b83eb442d57/users/af0baf4ed1c04420921c9d87aed78e85/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:50:50.610629 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4f12e-0e81-42a2-aa94-dc416c1d9174 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:50.649792 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a4f12e-0e81-42a2-aa94-dc416c1d9174 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:50.848224 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0818de-0d0b-4d5d-bd91-defc860b5987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:50.851396 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0818de-0d0b-4d5d-bd91-defc860b5987 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:50.853713 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0818de-0d0b-4d5d-bd91-defc860b5987 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:50.855688 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0818de-0d0b-4d5d-bd91-defc860b5987 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:50.859824 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0818de-0d0b-4d5d-bd91-defc860b5987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:50.862247 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0818de-0d0b-4d5d-bd91-defc860b5987 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:50.907061 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a719b9-f5a5-4984-b9f3-7bebcf08bd1b admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:50.908131 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 669/1332] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:50:50 2022] GET /identity/v3/roles => generated 1420 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:50:50.936913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a719b9-f5a5-4984-b9f3-7bebcf08bd1b admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:50.949907 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf071d69-2f47-4387-be86-a2e53ed5f58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:50.950611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf071d69-2f47-4387-be86-a2e53ed5f58f admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:50.950892 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf071d69-2f47-4387-be86-a2e53ed5f58f admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:50.951444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf071d69-2f47-4387-be86-a2e53ed5f58f admin admin] PATH_INFO: `/v3/projects/b1ab77d2ff914022bbdf3b83eb442d57/users/af0baf4ed1c04420921c9d87aed78e85/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:50.953455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf071d69-2f47-4387-be86-a2e53ed5f58f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1ab77d2ff914022bbdf3b83eb442d57, user_id=af0baf4ed1c04420921c9d87aed78e85, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:50.956834 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf071d69-2f47-4387-be86-a2e53ed5f58f admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:50.985054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 664/1333] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:50:50 2022] PUT /identity/v3/projects/b1ab77d2ff914022bbdf3b83eb442d57/users/af0baf4ed1c04420921c9d87aed78e85/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:50:50.999843 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff42791-6c54-4c3c-8ab5-bcfc24c5708a None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:51.000328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff42791-6c54-4c3c-8ab5-bcfc24c5708a None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:51.000489 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff42791-6c54-4c3c-8ab5-bcfc24c5708a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:51.085951 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-7ff42791-6c54-4c3c-8ab5-bcfc24c5708a None None] MFA Rules not processed for user `af0baf4ed1c04420921c9d87aed78e85`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:51.089369 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-7ff42791-6c54-4c3c-8ab5-bcfc24c5708a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:51.284629 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 670/1334] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:51.299407 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf071d69-2f47-4387-be86-a2e53ed5f58f admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:51.301226 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-bf071d69-2f47-4387-be86-a2e53ed5f58f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:51.522360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf071d69-2f47-4387-be86-a2e53ed5f58f admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:51.530461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe1ad43-04f8-4a08-9ba3-a687964c03e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:51.530461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1ad43-04f8-4a08-9ba3-a687964c03e3 service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:51.530461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1ad43-04f8-4a08-9ba3-a687964c03e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:51.530461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe1ad43-04f8-4a08-9ba3-a687964c03e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:51.544194 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe1ad43-04f8-4a08-9ba3-a687964c03e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:51.546300 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe1ad43-04f8-4a08-9ba3-a687964c03e3 service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:51.558775 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 665/1335] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:50:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:50:51.944511 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341cfd1-544b-4cfb-a0a8-ac07835e3cc1 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:51.945020 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341cfd1-544b-4cfb-a0a8-ac07835e3cc1 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:51.945105 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341cfd1-544b-4cfb-a0a8-ac07835e3cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:52.004470 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-f341cfd1-544b-4cfb-a0a8-ac07835e3cc1 None None] MFA Rules not processed for user `6f1245ffc45b4c99990e746068bd852c`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:52.005748 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-f341cfd1-544b-4cfb-a0a8-ac07835e3cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:52.034950 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:52.208026 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 671/1336] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:52.279206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe1ad43-04f8-4a08-9ba3-a687964c03e3 service cinder] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:52.286023 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-abe1ad43-04f8-4a08-9ba3-a687964c03e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:52.439896 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe1ad43-04f8-4a08-9ba3-a687964c03e3 service cinder] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:52.449335 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83ae80-10d0-405b-aed8-86e3c303dd44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:52.462946 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83ae80-10d0-405b-aed8-86e3c303dd44 service nova] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:52.464434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83ae80-10d0-405b-aed8-86e3c303dd44 service nova] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:52.465352 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d83ae80-10d0-405b-aed8-86e3c303dd44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:52.477637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d83ae80-10d0-405b-aed8-86e3c303dd44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:52.479877 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d83ae80-10d0-405b-aed8-86e3c303dd44 service nova] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:52.508554 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 666/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:50:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:50:53.474436 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341cfd1-544b-4cfb-a0a8-ac07835e3cc1 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:53.783768 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341cfd1-544b-4cfb-a0a8-ac07835e3cc1 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:53.795939 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7d77d-4d9a-4157-8140-7c8c586f7066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:53.796812 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7d77d-4d9a-4157-8140-7c8c586f7066 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:53.797370 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7d77d-4d9a-4157-8140-7c8c586f7066 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:53.797631 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7d77d-4d9a-4157-8140-7c8c586f7066 admin admin] PATH_INFO: `/v3/users/4e5cd48dc3b4421594da56091edf3367` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:53.802935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e7d77d-4d9a-4157-8140-7c8c586f7066 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e5cd48dc3b4421594da56091edf3367)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:53.804149 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: WARNING py.warnings [None req-f7e7d77d-4d9a-4157-8140-7c8c586f7066 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 03:50:53.804149 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: warnings.warn(msg) Oct 17 03:50:53.804149 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: Oct 17 03:50:53.810848 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e7d77d-4d9a-4157-8140-7c8c586f7066 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:53.832474 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83ae80-10d0-405b-aed8-86e3c303dd44 service nova] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:53.833265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-8d83ae80-10d0-405b-aed8-86e3c303dd44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:53.870389 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d83ae80-10d0-405b-aed8-86e3c303dd44 service nova] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:53.877215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c3a5ae-9fb7-48ef-8bc0-09bf3f354d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:53.878354 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3a5ae-9fb7-48ef-8bc0-09bf3f354d70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:53.879602 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3a5ae-9fb7-48ef-8bc0-09bf3f354d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:53.880266 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c3a5ae-9fb7-48ef-8bc0-09bf3f354d70 admin admin] PATH_INFO: `/v3/users/7619ccd38dbe408bbde7f354dcb32111` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:53.882636 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c3a5ae-9fb7-48ef-8bc0-09bf3f354d70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7619ccd38dbe408bbde7f354dcb32111)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:53.886134 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c3a5ae-9fb7-48ef-8bc0-09bf3f354d70 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:53.914015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:54.179854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-c3c3a5ae-9fb7-48ef-8bc0-09bf3f354d70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7619ccd38dbe408bbde7f354dcb32111'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:54.182430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-c3c3a5ae-9fb7-48ef-8bc0-09bf3f354d70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7619ccd38dbe408bbde7f354dcb32111'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:54.188375 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f7e7d77d-4d9a-4157-8140-7c8c586f7066 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e5cd48dc3b4421594da56091edf3367'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:54.279589 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-c3c3a5ae-9fb7-48ef-8bc0-09bf3f354d70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7619ccd38dbe408bbde7f354dcb32111'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:54.305802 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-c3c3a5ae-9fb7-48ef-8bc0-09bf3f354d70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7619ccd38dbe408bbde7f354dcb32111'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:54.306631 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-c3c3a5ae-9fb7-48ef-8bc0-09bf3f354d70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7619ccd38dbe408bbde7f354dcb32111'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:54.346399 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 667/1338] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:50:53 2022] DELETE /identity/v3/users/7619ccd38dbe408bbde7f354dcb32111 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:50:54.348110 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd4b60-7e60-43cc-9c6c-d222f4eb0cb7 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:54.348917 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd4b60-7e60-43cc-9c6c-d222f4eb0cb7 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:54.348917 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd4b60-7e60-43cc-9c6c-d222f4eb0cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:54.375994 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f7e7d77d-4d9a-4157-8140-7c8c586f7066 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e5cd48dc3b4421594da56091edf3367'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:54.399861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-dfcd4b60-7e60-43cc-9c6c-d222f4eb0cb7 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:54.401611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-dfcd4b60-7e60-43cc-9c6c-d222f4eb0cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:54.432575 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f7e7d77d-4d9a-4157-8140-7c8c586f7066 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e5cd48dc3b4421594da56091edf3367'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:54.439293 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f7e7d77d-4d9a-4157-8140-7c8c586f7066 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e5cd48dc3b4421594da56091edf3367'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:54.441863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f7e7d77d-4d9a-4157-8140-7c8c586f7066 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e5cd48dc3b4421594da56091edf3367'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:54.487977 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 668/1339] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:54.489675 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 672/1340] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:50:53 2022] DELETE /identity/v3/users/4e5cd48dc3b4421594da56091edf3367 => generated 0 bytes in 1015 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:50:54.500142 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797b6f7-a168-40a6-96c2-0ac0d0c6c19e None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:54.500561 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797b6f7-a168-40a6-96c2-0ac0d0c6c19e None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:54.500956 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797b6f7-a168-40a6-96c2-0ac0d0c6c19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:54.514540 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7d77d-4d9a-4157-8140-7c8c586f7066 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:54.516917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-f7e7d77d-4d9a-4157-8140-7c8c586f7066 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:54.580556 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-3797b6f7-a168-40a6-96c2-0ac0d0c6c19e None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:54.582642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-3797b6f7-a168-40a6-96c2-0ac0d0c6c19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:54.654259 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:54.734080 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7d77d-4d9a-4157-8140-7c8c586f7066 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:54.747807 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29270207-e97a-4979-aeaa-30672700708d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:54.750014 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29270207-e97a-4979-aeaa-30672700708d service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:54.750526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29270207-e97a-4979-aeaa-30672700708d service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:54.751310 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29270207-e97a-4979-aeaa-30672700708d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:54.783298 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 669/1341] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:54.816098 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797b6f7-a168-40a6-96c2-0ac0d0c6c19e None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:54.833779 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797b6f7-a168-40a6-96c2-0ac0d0c6c19e None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:54.840927 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82173d-54cd-41a9-954f-a6a8f2cc77f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:54.843901 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82173d-54cd-41a9-954f-a6a8f2cc77f4 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:54.843901 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82173d-54cd-41a9-954f-a6a8f2cc77f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:54.843901 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea82173d-54cd-41a9-954f-a6a8f2cc77f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:54.870647 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82173d-54cd-41a9-954f-a6a8f2cc77f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:54.876567 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea82173d-54cd-41a9-954f-a6a8f2cc77f4 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:54.906713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 670/1342] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:50:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:50:54.960788 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29270207-e97a-4979-aeaa-30672700708d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:54.960926 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29270207-e97a-4979-aeaa-30672700708d service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:55.010962 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 673/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:50:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:50:56.758528 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82173d-54cd-41a9-954f-a6a8f2cc77f4 service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:56.760305 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-ea82173d-54cd-41a9-954f-a6a8f2cc77f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:56.790384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea82173d-54cd-41a9-954f-a6a8f2cc77f4 service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:56.803840 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea0c8c-a03b-4926-a4b8-3164fcb73bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:56.804991 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea0c8c-a03b-4926-a4b8-3164fcb73bca admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:56.805285 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea0c8c-a03b-4926-a4b8-3164fcb73bca admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:56.805565 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea0c8c-a03b-4926-a4b8-3164fcb73bca admin admin] PATH_INFO: `/v3/users/15dac03b55094e93ac1d567c9c8f7bb3` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:56.811881 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea0c8c-a03b-4926-a4b8-3164fcb73bca admin admin] RBAC: Authorizing `identity:delete_user(user_id=15dac03b55094e93ac1d567c9c8f7bb3)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:56.813882 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea0c8c-a03b-4926-a4b8-3164fcb73bca admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:56.859314 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:56.984412 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9680682-a24b-4872-81c4-95ecb68849e4 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:56.985325 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9680682-a24b-4872-81c4-95ecb68849e4 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:56.985739 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9680682-a24b-4872-81c4-95ecb68849e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:57.083607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-c9680682-a24b-4872-81c4-95ecb68849e4 None None] MFA Rules not processed for user `6307c4e28ee044dab595913514501537`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:57.092668 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c9680682-a24b-4872-81c4-95ecb68849e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:57.147456 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-95ea0c8c-a03b-4926-a4b8-3164fcb73bca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15dac03b55094e93ac1d567c9c8f7bb3'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:57.149620 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-95ea0c8c-a03b-4926-a4b8-3164fcb73bca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15dac03b55094e93ac1d567c9c8f7bb3'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:57.233916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 674/1344] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:50:57.235055 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9680682-a24b-4872-81c4-95ecb68849e4 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:57.238671 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c9680682-a24b-4872-81c4-95ecb68849e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:57.244836 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-95ea0c8c-a03b-4926-a4b8-3164fcb73bca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15dac03b55094e93ac1d567c9c8f7bb3'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:57.268184 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9680682-a24b-4872-81c4-95ecb68849e4 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:57.275340 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442d1de-10ae-4bf7-bd71-56ee115765ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:57.276770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442d1de-10ae-4bf7-bd71-56ee115765ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:57.277214 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442d1de-10ae-4bf7-bd71-56ee115765ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:57.277705 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442d1de-10ae-4bf7-bd71-56ee115765ff admin admin] PATH_INFO: `/v3/projects/84929e7da65d4696913f3fc844f90854` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:57.279421 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442d1de-10ae-4bf7-bd71-56ee115765ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=84929e7da65d4696913f3fc844f90854)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:57.281844 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442d1de-10ae-4bf7-bd71-56ee115765ff admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:57.289503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-95ea0c8c-a03b-4926-a4b8-3164fcb73bca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15dac03b55094e93ac1d567c9c8f7bb3'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:57.294665 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-95ea0c8c-a03b-4926-a4b8-3164fcb73bca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15dac03b55094e93ac1d567c9c8f7bb3'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:57.325029 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 671/1345] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:50:56 2022] DELETE /identity/v3/users/15dac03b55094e93ac1d567c9c8f7bb3 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:50:57.327098 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f293045-91f7-4d74-845c-11c47f34a6ac None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:57.327444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f293045-91f7-4d74-845c-11c47f34a6ac None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:57.327827 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f293045-91f7-4d74-845c-11c47f34a6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:57.375607 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-1f293045-91f7-4d74-845c-11c47f34a6ac None None] MFA Rules not processed for user `6307c4e28ee044dab595913514501537`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:57.377530 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-1f293045-91f7-4d74-845c-11c47f34a6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:57.499608 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 672/1346] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:57.500964 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f240d0-fc9e-4b88-8466-87a6bb397fcf None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:57.501180 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f240d0-fc9e-4b88-8466-87a6bb397fcf None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:57.501577 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f240d0-fc9e-4b88-8466-87a6bb397fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:57.525636 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-7442d1de-10ae-4bf7-bd71-56ee115765ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84929e7da65d4696913f3fc844f90854', 'request_id': 'req-7442d1de-10ae-4bf7-bd71-56ee115765ff'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:57.544214 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-06f240d0-fc9e-4b88-8466-87a6bb397fcf None None] MFA Rules not processed for user `ec2ca63a9f4c4c18bde25b1e4eab93ab`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:57.545873 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-06f240d0-fc9e-4b88-8466-87a6bb397fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:57.621560 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-7442d1de-10ae-4bf7-bd71-56ee115765ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84929e7da65d4696913f3fc844f90854', 'request_id': 'req-7442d1de-10ae-4bf7-bd71-56ee115765ff'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:57.629708 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-7442d1de-10ae-4bf7-bd71-56ee115765ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84929e7da65d4696913f3fc844f90854', 'request_id': 'req-7442d1de-10ae-4bf7-bd71-56ee115765ff'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:57.633181 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:57.658685 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-7442d1de-10ae-4bf7-bd71-56ee115765ff admin admin] The token cache is being invalidate because project 84929e7da65d4696913f3fc844f90854 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:50:57.662449 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-7442d1de-10ae-4bf7-bd71-56ee115765ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:57.663683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-7442d1de-10ae-4bf7-bd71-56ee115765ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:57.668058 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 675/1347] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:50:57 2022] DELETE /identity/v3/projects/84929e7da65d4696913f3fc844f90854 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:50:57.669806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffc24f-a404-4b31-9c2c-348820fe3ed9 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:57.673889 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffc24f-a404-4b31-9c2c-348820fe3ed9 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:57.674440 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ffc24f-a404-4b31-9c2c-348820fe3ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:57.685488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 673/1348] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:57.697199 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f240d0-fc9e-4b88-8466-87a6bb397fcf None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:57.709716 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-06f240d0-fc9e-4b88-8466-87a6bb397fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:57.734838 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-02ffc24f-a404-4b31-9c2c-348820fe3ed9 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:57.737661 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-02ffc24f-a404-4b31-9c2c-348820fe3ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:57.890088 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f240d0-fc9e-4b88-8466-87a6bb397fcf None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:57.893670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 676/1349] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:57.893670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffc24f-a404-4b31-9c2c-348820fe3ed9 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:57.894534 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-02ffc24f-a404-4b31-9c2c-348820fe3ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:57.920663 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9c3867-c5b8-448a-ad1a-a2efb69c1ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:57.921486 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c3867-c5b8-448a-ad1a-a2efb69c1ca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:57.923440 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c3867-c5b8-448a-ad1a-a2efb69c1ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:57.923440 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c3867-c5b8-448a-ad1a-a2efb69c1ca8 admin admin] PATH_INFO: `/v3/projects/af361498dd7b4973b66b005b255b3828` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:57.927618 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9c3867-c5b8-448a-ad1a-a2efb69c1ca8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af361498dd7b4973b66b005b255b3828)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:57.928798 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: WARNING py.warnings [None req-1d9c3867-c5b8-448a-ad1a-a2efb69c1ca8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 17 03:50:57.928798 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: warnings.warn(msg) Oct 17 03:50:57.928798 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: Oct 17 03:50:57.930014 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9c3867-c5b8-448a-ad1a-a2efb69c1ca8 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:58.026311 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ffc24f-a404-4b31-9c2c-348820fe3ed9 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:58.033396 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d34c0-37fc-4fc8-b7a3-3a37564dcae6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:58.034120 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d34c0-37fc-4fc8-b7a3-3a37564dcae6 service nova] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:58.034458 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d34c0-37fc-4fc8-b7a3-3a37564dcae6 service nova] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:58.034738 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d34c0-37fc-4fc8-b7a3-3a37564dcae6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:58.036640 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-0a3d34c0-37fc-4fc8-b7a3-3a37564dcae6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:58.138109 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-1d9c3867-c5b8-448a-ad1a-a2efb69c1ca8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af361498dd7b4973b66b005b255b3828', 'request_id': 'req-1d9c3867-c5b8-448a-ad1a-a2efb69c1ca8'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:58.142714 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d34c0-37fc-4fc8-b7a3-3a37564dcae6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:58.144971 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d34c0-37fc-4fc8-b7a3-3a37564dcae6 service nova] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:58.166039 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 677/1350] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:50:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:50:58.167467 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b046f-b03c-415b-ade6-c657300a30ca None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:58.167827 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b046f-b03c-415b-ade6-c657300a30ca None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:58.168086 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b046f-b03c-415b-ade6-c657300a30ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:58.209514 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-1d9c3867-c5b8-448a-ad1a-a2efb69c1ca8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af361498dd7b4973b66b005b255b3828', 'request_id': 'req-1d9c3867-c5b8-448a-ad1a-a2efb69c1ca8'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:58.229108 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-1d9c3867-c5b8-448a-ad1a-a2efb69c1ca8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af361498dd7b4973b66b005b255b3828', 'request_id': 'req-1d9c3867-c5b8-448a-ad1a-a2efb69c1ca8'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:58.273232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-962b046f-b03c-415b-ade6-c657300a30ca None None] MFA Rules not processed for user `57ee3a13e33549c9a90ebde353fcd6f2`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:58.273740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-1d9c3867-c5b8-448a-ad1a-a2efb69c1ca8 admin admin] The token cache is being invalidate because project af361498dd7b4973b66b005b255b3828 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:50:58.274150 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-1d9c3867-c5b8-448a-ad1a-a2efb69c1ca8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:58.274474 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-962b046f-b03c-415b-ade6-c657300a30ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:58.274912 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-1d9c3867-c5b8-448a-ad1a-a2efb69c1ca8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:58.280358 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 674/1351] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:50:57 2022] DELETE /identity/v3/projects/af361498dd7b4973b66b005b255b3828 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:50:58.285783 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c0f62-74be-4ed3-be02-883a37af13a8 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:58.285783 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c0f62-74be-4ed3-be02-883a37af13a8 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:58.285783 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c0f62-74be-4ed3-be02-883a37af13a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:58.325072 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-de9c0f62-74be-4ed3-be02-883a37af13a8 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:58.327272 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-de9c0f62-74be-4ed3-be02-883a37af13a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:58.331706 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:58.457390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 678/1352] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:58 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:58.457390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731a11e-07a9-4b3d-ad0b-b175df24dc9d None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:58.457390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731a11e-07a9-4b3d-ad0b-b175df24dc9d None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:58.457390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731a11e-07a9-4b3d-ad0b-b175df24dc9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:58.502036 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 675/1353] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:58.502976 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c0f62-74be-4ed3-be02-883a37af13a8 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:58.509303 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-de9c0f62-74be-4ed3-be02-883a37af13a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:58.528486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-0731a11e-07a9-4b3d-ad0b-b175df24dc9d None None] MFA Rules not processed for user `ec2ca63a9f4c4c18bde25b1e4eab93ab`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:58.529892 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-0731a11e-07a9-4b3d-ad0b-b175df24dc9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:58.608120 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:58.668674 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c0f62-74be-4ed3-be02-883a37af13a8 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:58.680731 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add75ffe-30f5-4e61-9050-aa7333b4ad08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:58.680731 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add75ffe-30f5-4e61-9050-aa7333b4ad08 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:58.680731 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add75ffe-30f5-4e61-9050-aa7333b4ad08 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:58.680731 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add75ffe-30f5-4e61-9050-aa7333b4ad08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:58.682535 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-add75ffe-30f5-4e61-9050-aa7333b4ad08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:58.683164 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 679/1354] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:58 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:58.684061 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731a11e-07a9-4b3d-ad0b-b175df24dc9d None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:58.685346 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-0731a11e-07a9-4b3d-ad0b-b175df24dc9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:58.710674 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add75ffe-30f5-4e61-9050-aa7333b4ad08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:58.711962 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add75ffe-30f5-4e61-9050-aa7333b4ad08 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:58.729069 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731a11e-07a9-4b3d-ad0b-b175df24dc9d None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:58.732643 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 676/1355] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:50:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:50:58.733603 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add75ffe-30f5-4e61-9050-aa7333b4ad08 service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:58.734717 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378db9e-91f3-4861-8e53-8c69b8d4877a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:58.735466 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378db9e-91f3-4861-8e53-8c69b8d4877a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:58.735771 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378db9e-91f3-4861-8e53-8c69b8d4877a admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:58.736051 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378db9e-91f3-4861-8e53-8c69b8d4877a admin admin] PATH_INFO: `/v3/users/af0baf4ed1c04420921c9d87aed78e85` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:58.736609 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-add75ffe-30f5-4e61-9050-aa7333b4ad08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:58.747124 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378db9e-91f3-4861-8e53-8c69b8d4877a admin admin] RBAC: Authorizing `identity:delete_user(user_id=af0baf4ed1c04420921c9d87aed78e85)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:58.749671 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378db9e-91f3-4861-8e53-8c69b8d4877a admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:58.782747 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:58.926687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add75ffe-30f5-4e61-9050-aa7333b4ad08 service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:58.942626 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc77f40f-93e3-4aa6-85cf-b62862403d1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:58.945130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77f40f-93e3-4aa6-85cf-b62862403d1a service nova] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:58.945130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77f40f-93e3-4aa6-85cf-b62862403d1a service nova] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:58.945130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77f40f-93e3-4aa6-85cf-b62862403d1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:58.963840 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc77f40f-93e3-4aa6-85cf-b62862403d1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:58.978344 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc77f40f-93e3-4aa6-85cf-b62862403d1a service nova] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:59.022952 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 677/1356] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:50:58 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:50:59.026934 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ba37d-2f07-4d03-b5bb-a5bbc9dcca10 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:59.029381 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ba37d-2f07-4d03-b5bb-a5bbc9dcca10 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:59.029666 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093ba37d-2f07-4d03-b5bb-a5bbc9dcca10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:59.040237 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-1378db9e-91f3-4861-8e53-8c69b8d4877a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af0baf4ed1c04420921c9d87aed78e85'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:59.077325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-093ba37d-2f07-4d03-b5bb-a5bbc9dcca10 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:59.078562 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-093ba37d-2f07-4d03-b5bb-a5bbc9dcca10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:59.101219 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 678/1357] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:59.107383 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd380ce8-ec48-4ecf-aee0-172787af6d52 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:59.111858 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd380ce8-ec48-4ecf-aee0-172787af6d52 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:59.112265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd380ce8-ec48-4ecf-aee0-172787af6d52 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:59.116232 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 679/1358] 10.222.0.17 () {54 vars in 912 bytes} [Mon Oct 17 03:50:59 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:50:59.120526 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd380ce8-ec48-4ecf-aee0-172787af6d52 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:59.136085 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-1378db9e-91f3-4861-8e53-8c69b8d4877a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af0baf4ed1c04420921c9d87aed78e85'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:59.152211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd380ce8-ec48-4ecf-aee0-172787af6d52 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:59.165771 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-1378db9e-91f3-4861-8e53-8c69b8d4877a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af0baf4ed1c04420921c9d87aed78e85'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:59.167494 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5581e8b-be25-4e6d-98c9-a33ceeec7e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:59.167964 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-1378db9e-91f3-4861-8e53-8c69b8d4877a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af0baf4ed1c04420921c9d87aed78e85'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:59.168445 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5581e8b-be25-4e6d-98c9-a33ceeec7e6c admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:59.168736 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5581e8b-be25-4e6d-98c9-a33ceeec7e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:59.169040 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5581e8b-be25-4e6d-98c9-a33ceeec7e6c admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:59.169881 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5581e8b-be25-4e6d-98c9-a33ceeec7e6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:50:59.169980 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-1378db9e-91f3-4861-8e53-8c69b8d4877a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af0baf4ed1c04420921c9d87aed78e85'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:50:59.170250 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5581e8b-be25-4e6d-98c9-a33ceeec7e6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:59.172657 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5581e8b-be25-4e6d-98c9-a33ceeec7e6c admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:59.195719 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 680/1359] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:50:59.197337 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bdf93-4245-495d-8f3c-7f55332eeed5 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:59.198843 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bdf93-4245-495d-8f3c-7f55332eeed5 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:59.199135 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747bdf93-4245-495d-8f3c-7f55332eeed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:59.209240 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 680/1360] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:50:58 2022] DELETE /identity/v3/users/af0baf4ed1c04420921c9d87aed78e85 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:50:59.209240 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378db9e-91f3-4861-8e53-8c69b8d4877a admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:59.209240 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-1378db9e-91f3-4861-8e53-8c69b8d4877a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:59.283630 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-747bdf93-4245-495d-8f3c-7f55332eeed5 None None] MFA Rules not processed for user `414707c545f641d3935720356b326672`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:59.289534 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-747bdf93-4245-495d-8f3c-7f55332eeed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:59.343461 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:59.397734 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378db9e-91f3-4861-8e53-8c69b8d4877a admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:59.413477 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503e9e7-b8c1-4407-a60b-5510c8decf48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:59.414605 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503e9e7-b8c1-4407-a60b-5510c8decf48 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:59.415081 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503e9e7-b8c1-4407-a60b-5510c8decf48 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:59.415526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503e9e7-b8c1-4407-a60b-5510c8decf48 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:59.416761 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503e9e7-b8c1-4407-a60b-5510c8decf48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:59.419947 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503e9e7-b8c1-4407-a60b-5510c8decf48 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:59.422784 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 681/1361] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Oct 17 03:50:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:50:59.423888 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747bdf93-4245-495d-8f3c-7f55332eeed5 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:59.425668 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-747bdf93-4245-495d-8f3c-7f55332eeed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:59.510950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:50:59.530754 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747bdf93-4245-495d-8f3c-7f55332eeed5 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:59.538541 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79912e3e-1e7e-45f6-90f0-10ff417ade0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:59.539648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79912e3e-1e7e-45f6-90f0-10ff417ade0f service nova] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:59.540133 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79912e3e-1e7e-45f6-90f0-10ff417ade0f service nova] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:59.541331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79912e3e-1e7e-45f6-90f0-10ff417ade0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:59.543832 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-79912e3e-1e7e-45f6-90f0-10ff417ade0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:59.644493 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 681/1362] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:50:59 2022] POST /identity/v3/projects => generated 381 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:50:59.645223 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503e9e7-b8c1-4407-a60b-5510c8decf48 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:59.673556 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-6503e9e7-b8c1-4407-a60b-5510c8decf48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:59.738355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79912e3e-1e7e-45f6-90f0-10ff417ade0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:59.740035 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79912e3e-1e7e-45f6-90f0-10ff417ade0f service nova] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:59.764673 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 682/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:50:59 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:50:59.765522 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79912e3e-1e7e-45f6-90f0-10ff417ade0f service nova] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:50:59.788040 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79912e3e-1e7e-45f6-90f0-10ff417ade0f service nova] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:59.837643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503e9e7-b8c1-4407-a60b-5510c8decf48 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:50:59.844320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e4b90-0f73-4548-b6a5-0782e1e11cfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:59.844970 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e4b90-0f73-4548-b6a5-0782e1e11cfb service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:59.845258 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e4b90-0f73-4548-b6a5-0782e1e11cfb service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:59.845587 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e4b90-0f73-4548-b6a5-0782e1e11cfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:59.862861 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e4b90-0f73-4548-b6a5-0782e1e11cfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:59.866258 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e4b90-0f73-4548-b6a5-0782e1e11cfb service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:50:59.885145 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 682/1364] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:50:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:50:59.886586 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f04ca8-0545-48b9-be5f-86145c578013 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:59.886874 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f04ca8-0545-48b9-be5f-86145c578013 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:59.887150 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f04ca8-0545-48b9-be5f-86145c578013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:59.918960 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-81f04ca8-0545-48b9-be5f-86145c578013 None None] MFA Rules not processed for user `6f1245ffc45b4c99990e746068bd852c`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:50:59.920313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-81f04ca8-0545-48b9-be5f-86145c578013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:50:59.944188 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4508747-28db-41cf-8137-7b2d0d05a542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:50:59.946894 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4508747-28db-41cf-8137-7b2d0d05a542 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:50:59.947353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4508747-28db-41cf-8137-7b2d0d05a542 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:50:59.947847 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4508747-28db-41cf-8137-7b2d0d05a542 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:50:59.949172 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4508747-28db-41cf-8137-7b2d0d05a542 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:50:59.953823 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4508747-28db-41cf-8137-7b2d0d05a542 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:00.014827 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 683/1365] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:50:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:00.016021 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f04ca8-0545-48b9-be5f-86145c578013 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:00.017678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-81f04ca8-0545-48b9-be5f-86145c578013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:00.040291 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f04ca8-0545-48b9-be5f-86145c578013 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:00.053094 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cadbde-7cbe-44fe-83ac-f2a0b71c318f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:00.053622 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cadbde-7cbe-44fe-83ac-f2a0b71c318f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:00.053904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cadbde-7cbe-44fe-83ac-f2a0b71c318f admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:00.054186 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cadbde-7cbe-44fe-83ac-f2a0b71c318f admin admin] PATH_INFO: `/v3/projects/f9eae63ad8f14fc9a6432a5c846981de` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:00.057316 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cadbde-7cbe-44fe-83ac-f2a0b71c318f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9eae63ad8f14fc9a6432a5c846981de)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:00.061973 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cadbde-7cbe-44fe-83ac-f2a0b71c318f admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:00.093337 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 683/1366] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:50:59 2022] POST /identity/v3/users => generated 291 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:00.098387 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4508747-28db-41cf-8137-7b2d0d05a542 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:00.146013 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4508747-28db-41cf-8137-7b2d0d05a542 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:00.167775 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a164451-a488-4730-956b-8ebaa574b5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:00.172601 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a164451-a488-4730-956b-8ebaa574b5ff admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:00.173092 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a164451-a488-4730-956b-8ebaa574b5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:00.177208 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a164451-a488-4730-956b-8ebaa574b5ff admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:00.178176 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a164451-a488-4730-956b-8ebaa574b5ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:00.187959 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a164451-a488-4730-956b-8ebaa574b5ff admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:00.223360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 684/1367] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:00.227657 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a164451-a488-4730-956b-8ebaa574b5ff admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:00.256511 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a164451-a488-4730-956b-8ebaa574b5ff admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:00.274413 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c0cadbde-7cbe-44fe-83ac-f2a0b71c318f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9eae63ad8f14fc9a6432a5c846981de', 'request_id': 'req-c0cadbde-7cbe-44fe-83ac-f2a0b71c318f'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:00.354169 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c0cadbde-7cbe-44fe-83ac-f2a0b71c318f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9eae63ad8f14fc9a6432a5c846981de', 'request_id': 'req-c0cadbde-7cbe-44fe-83ac-f2a0b71c318f'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:00.365963 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c0cadbde-7cbe-44fe-83ac-f2a0b71c318f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9eae63ad8f14fc9a6432a5c846981de', 'request_id': 'req-c0cadbde-7cbe-44fe-83ac-f2a0b71c318f'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:00.369830 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:00.372411 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108a2a9-8fdc-4f5a-976a-f133abe9f10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:00.373225 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108a2a9-8fdc-4f5a-976a-f133abe9f10b admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:00.373617 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108a2a9-8fdc-4f5a-976a-f133abe9f10b admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:00.374008 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108a2a9-8fdc-4f5a-976a-f133abe9f10b admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:00.375279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6108a2a9-8fdc-4f5a-976a-f133abe9f10b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:00.377132 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6108a2a9-8fdc-4f5a-976a-f133abe9f10b admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:00.390888 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c0cadbde-7cbe-44fe-83ac-f2a0b71c318f admin admin] The token cache is being invalidate because project f9eae63ad8f14fc9a6432a5c846981de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:00.391189 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c0cadbde-7cbe-44fe-83ac-f2a0b71c318f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:00.393699 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c0cadbde-7cbe-44fe-83ac-f2a0b71c318f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:00.399223 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 685/1368] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:51:00.400294 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108a2a9-8fdc-4f5a-976a-f133abe9f10b admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:00.402558 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6108a2a9-8fdc-4f5a-976a-f133abe9f10b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:00.404633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 684/1369] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:00 2022] DELETE /identity/v3/projects/f9eae63ad8f14fc9a6432a5c846981de => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:00.409526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cca256-a7dc-4a4f-a654-20b02c265693 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:00.417633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cca256-a7dc-4a4f-a654-20b02c265693 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:00.417633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cca256-a7dc-4a4f-a654-20b02c265693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:00.424886 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108a2a9-8fdc-4f5a-976a-f133abe9f10b admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:00.431685 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c3519f-48be-46d2-bc48-6085fcc4b163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:00.432537 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3519f-48be-46d2-bc48-6085fcc4b163 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:00.437956 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3519f-48be-46d2-bc48-6085fcc4b163 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:00.437956 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3519f-48be-46d2-bc48-6085fcc4b163 admin admin] PATH_INFO: `/v3/projects/2d6cba3edebd4f0eb5343491d5102c17/users/201facab38314abca6062b948dfd1c02/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:00.445763 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:00.464670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-05cca256-a7dc-4a4f-a654-20b02c265693 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:00.466245 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-05cca256-a7dc-4a4f-a654-20b02c265693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:00.469496 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c3519f-48be-46d2-bc48-6085fcc4b163 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d6cba3edebd4f0eb5343491d5102c17, user_id=201facab38314abca6062b948dfd1c02, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:00.475541 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c3519f-48be-46d2-bc48-6085fcc4b163 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:00.483584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 685/1370] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:00.484902 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cca256-a7dc-4a4f-a654-20b02c265693 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:00.492339 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-05cca256-a7dc-4a4f-a654-20b02c265693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:00.507452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cca256-a7dc-4a4f-a654-20b02c265693 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:00.513502 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32ddcd2-22ee-4b97-8ca8-56f9f0c4fb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:00.515527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ddcd2-22ee-4b97-8ca8-56f9f0c4fb0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:00.516110 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ddcd2-22ee-4b97-8ca8-56f9f0c4fb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:00.516562 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32ddcd2-22ee-4b97-8ca8-56f9f0c4fb0f admin admin] PATH_INFO: `/v3/projects/b1ab77d2ff914022bbdf3b83eb442d57` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:00.518674 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ddcd2-22ee-4b97-8ca8-56f9f0c4fb0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1ab77d2ff914022bbdf3b83eb442d57)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:00.521176 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32ddcd2-22ee-4b97-8ca8-56f9f0c4fb0f admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:00.542693 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 686/1371] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:00 2022] PUT /identity/v3/projects/2d6cba3edebd4f0eb5343491d5102c17/users/201facab38314abca6062b948dfd1c02/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:00.544278 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5288c-e113-48a4-b135-eae7e3f58b77 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:00.544565 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5288c-e113-48a4-b135-eae7e3f58b77 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:00.544837 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5288c-e113-48a4-b135-eae7e3f58b77 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:00.548124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 687/1372] 10.222.0.17 () {54 vars in 912 bytes} [Mon Oct 17 03:51:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:51:00.559192 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fa5e2-81ac-40e4-bb99-72a072934785 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:00.559476 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fa5e2-81ac-40e4-bb99-72a072934785 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:00.560347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fa5e2-81ac-40e4-bb99-72a072934785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:00.590183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-223fa5e2-81ac-40e4-bb99-72a072934785 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:00.591623 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-223fa5e2-81ac-40e4-bb99-72a072934785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:00.662471 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 688/1373] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:00.664076 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aea4a4-ed55-4868-a188-829f4a7932f2 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:00.664495 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aea4a4-ed55-4868-a188-829f4a7932f2 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:00.665892 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aea4a4-ed55-4868-a188-829f4a7932f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:00.680545 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f32ddcd2-22ee-4b97-8ca8-56f9f0c4fb0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1ab77d2ff914022bbdf3b83eb442d57', 'request_id': 'req-f32ddcd2-22ee-4b97-8ca8-56f9f0c4fb0f'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:00.731268 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-05aea4a4-ed55-4868-a188-829f4a7932f2 None None] MFA Rules not processed for user `201facab38314abca6062b948dfd1c02`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:00.734292 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-05aea4a4-ed55-4868-a188-829f4a7932f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:00.746642 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f32ddcd2-22ee-4b97-8ca8-56f9f0c4fb0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1ab77d2ff914022bbdf3b83eb442d57', 'request_id': 'req-f32ddcd2-22ee-4b97-8ca8-56f9f0c4fb0f'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:00.753282 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f32ddcd2-22ee-4b97-8ca8-56f9f0c4fb0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1ab77d2ff914022bbdf3b83eb442d57', 'request_id': 'req-f32ddcd2-22ee-4b97-8ca8-56f9f0c4fb0f'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:00.770211 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f32ddcd2-22ee-4b97-8ca8-56f9f0c4fb0f admin admin] The token cache is being invalidate because project b1ab77d2ff914022bbdf3b83eb442d57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:00.770572 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f32ddcd2-22ee-4b97-8ca8-56f9f0c4fb0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:00.771266 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f32ddcd2-22ee-4b97-8ca8-56f9f0c4fb0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:00.773018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 686/1374] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:00 2022] DELETE /identity/v3/projects/b1ab77d2ff914022bbdf3b83eb442d57 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:00.778382 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09902179-f78f-4499-8fe7-316dd16054a4 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:00.778849 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09902179-f78f-4499-8fe7-316dd16054a4 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:00.779143 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09902179-f78f-4499-8fe7-316dd16054a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:00.834992 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-09902179-f78f-4499-8fe7-316dd16054a4 None None] MFA Rules not processed for user `414707c545f641d3935720356b326672`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:00.836402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-09902179-f78f-4499-8fe7-316dd16054a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:00.864760 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 689/1375] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:00.866154 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aea4a4-ed55-4868-a188-829f4a7932f2 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:00.870229 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-05aea4a4-ed55-4868-a188-829f4a7932f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:00.926288 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 687/1376] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Oct 17 03:51:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:00.928235 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40ec85-0f19-44a2-ae2b-f7e43d0906d2 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:00.928532 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40ec85-0f19-44a2-ae2b-f7e43d0906d2 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:00.928808 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40ec85-0f19-44a2-ae2b-f7e43d0906d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:01.001538 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-7c40ec85-0f19-44a2-ae2b-f7e43d0906d2 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:01.010235 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-7c40ec85-0f19-44a2-ae2b-f7e43d0906d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:01.032163 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aea4a4-ed55-4868-a188-829f4a7932f2 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:01.036247 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 688/1377] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:01.038205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40ec85-0f19-44a2-ae2b-f7e43d0906d2 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:01.043501 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-7c40ec85-0f19-44a2-ae2b-f7e43d0906d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:01.044851 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe01ea-47e6-4da2-89f1-947c16a247db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:01.047017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe01ea-47e6-4da2-89f1-947c16a247db admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:01.047017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe01ea-47e6-4da2-89f1-947c16a247db admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:01.047278 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe01ea-47e6-4da2-89f1-947c16a247db admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:01.047975 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe01ea-47e6-4da2-89f1-947c16a247db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:51:01.048280 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe01ea-47e6-4da2-89f1-947c16a247db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:01.051363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe01ea-47e6-4da2-89f1-947c16a247db admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:01.090154 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 690/1378] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:51:01.091075 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe01ea-47e6-4da2-89f1-947c16a247db admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:01.094379 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-fbbe01ea-47e6-4da2-89f1-947c16a247db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:01.121519 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe01ea-47e6-4da2-89f1-947c16a247db admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:01.127620 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1d9ef-0d47-4049-a6ff-0871c81d5043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:01.128425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1d9ef-0d47-4049-a6ff-0871c81d5043 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:01.128852 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1d9ef-0d47-4049-a6ff-0871c81d5043 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:01.129554 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc1d9ef-0d47-4049-a6ff-0871c81d5043 admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:01.129857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc1d9ef-0d47-4049-a6ff-0871c81d5043 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:01.131905 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc1d9ef-0d47-4049-a6ff-0871c81d5043 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:01.210198 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40ec85-0f19-44a2-ae2b-f7e43d0906d2 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:01.221172 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a747c-9e8c-44d7-8e0b-4035ba9a41ed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b500a113a2aa4a95875d7e094b917db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:01.221172 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a747c-9e8c-44d7-8e0b-4035ba9a41ed service placement] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:01.221172 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a747c-9e8c-44d7-8e0b-4035ba9a41ed service placement] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:01.221172 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a747c-9e8c-44d7-8e0b-4035ba9a41ed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:01.226461 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-674a747c-9e8c-44d7-8e0b-4035ba9a41ed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:01.242956 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a747c-9e8c-44d7-8e0b-4035ba9a41ed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:01.261650 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a747c-9e8c-44d7-8e0b-4035ba9a41ed service placement] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:01.278280 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 691/1379] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:01 2022] POST /identity/v3/projects => generated 381 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:01.279376 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1d9ef-0d47-4049-a6ff-0871c81d5043 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:01.283116 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-ecc1d9ef-0d47-4049-a6ff-0871c81d5043 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:01.284102 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 689/1380] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:01.290834 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a70828-8f84-417c-8285-236a83167840 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:01.292356 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a70828-8f84-417c-8285-236a83167840 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:01.292809 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a70828-8f84-417c-8285-236a83167840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:01.325960 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-c6a70828-8f84-417c-8285-236a83167840 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:01.330593 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c6a70828-8f84-417c-8285-236a83167840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:01.359102 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:01.422442 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc1d9ef-0d47-4049-a6ff-0871c81d5043 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:01.434346 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee9af6-cd40-4544-8e2f-099c1cb9dca7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:01.435318 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee9af6-cd40-4544-8e2f-099c1cb9dca7 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:01.435854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee9af6-cd40-4544-8e2f-099c1cb9dca7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:01.436198 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee9af6-cd40-4544-8e2f-099c1cb9dca7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:01.469089 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee9af6-cd40-4544-8e2f-099c1cb9dca7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:01.470564 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee9af6-cd40-4544-8e2f-099c1cb9dca7 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:01.486388 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 690/1381] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:01.487576 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a70828-8f84-417c-8285-236a83167840 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:01.498183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 692/1382] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:01.499436 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee9af6-cd40-4544-8e2f-099c1cb9dca7 service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:01.511974 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a70828-8f84-417c-8285-236a83167840 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:01.523028 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21bd329-540f-4ebe-8cae-4ab63e1987b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:01.523676 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21bd329-540f-4ebe-8cae-4ab63e1987b5 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:01.524145 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21bd329-540f-4ebe-8cae-4ab63e1987b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:01.524442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21bd329-540f-4ebe-8cae-4ab63e1987b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:01.525178 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21bd329-540f-4ebe-8cae-4ab63e1987b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:01.530179 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21bd329-540f-4ebe-8cae-4ab63e1987b5 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:01.562232 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee9af6-cd40-4544-8e2f-099c1cb9dca7 service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:01.571815 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb5e2a-93ae-4e0e-a85b-3a0eef607467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:01.572955 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb5e2a-93ae-4e0e-a85b-3a0eef607467 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:01.574387 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb5e2a-93ae-4e0e-a85b-3a0eef607467 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:01.574744 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb5e2a-93ae-4e0e-a85b-3a0eef607467 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:01.575532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb5e2a-93ae-4e0e-a85b-3a0eef607467 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:01.580250 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb5e2a-93ae-4e0e-a85b-3a0eef607467 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:01.619243 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 691/1383] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:01 2022] POST /identity/v3/projects => generated 365 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:01.620226 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21bd329-540f-4ebe-8cae-4ab63e1987b5 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:01.733918 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 693/1384] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:01 2022] POST /identity/v3/users => generated 297 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:01.734981 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21bd329-540f-4ebe-8cae-4ab63e1987b5 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:01.735289 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb5e2a-93ae-4e0e-a85b-3a0eef607467 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:01.740853 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25d60b-5f36-4270-9b83-5a792aa13895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:01.741532 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25d60b-5f36-4270-9b83-5a792aa13895 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:01.741936 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25d60b-5f36-4270-9b83-5a792aa13895 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:01.742155 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed25d60b-5f36-4270-9b83-5a792aa13895 admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:01.742916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed25d60b-5f36-4270-9b83-5a792aa13895 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:51:01.743230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed25d60b-5f36-4270-9b83-5a792aa13895 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:01.745245 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed25d60b-5f36-4270-9b83-5a792aa13895 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:01.749306 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb5e2a-93ae-4e0e-a85b-3a0eef607467 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:01.754922 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5285c6-0299-4984-b0e0-9f3da3e34dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:01.756415 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5285c6-0299-4984-b0e0-9f3da3e34dc9 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:01.757172 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5285c6-0299-4984-b0e0-9f3da3e34dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:01.757587 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5285c6-0299-4984-b0e0-9f3da3e34dc9 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:01.758401 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5285c6-0299-4984-b0e0-9f3da3e34dc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:01.760792 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5285c6-0299-4984-b0e0-9f3da3e34dc9 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:01.762288 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 692/1385] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:51:01.764198 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25d60b-5f36-4270-9b83-5a792aa13895 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:01.767157 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:01.781922 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed25d60b-5f36-4270-9b83-5a792aa13895 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:01.788425 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee2714-97a7-442e-801e-02958b2312e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:01.789253 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee2714-97a7-442e-801e-02958b2312e1 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:01.791927 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee2714-97a7-442e-801e-02958b2312e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:01.793128 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee2714-97a7-442e-801e-02958b2312e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:01.798794 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee2714-97a7-442e-801e-02958b2312e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:01.798794 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee2714-97a7-442e-801e-02958b2312e1 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:01.831245 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 693/1386] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:01.832556 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee2714-97a7-442e-801e-02958b2312e1 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:01.848884 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee2714-97a7-442e-801e-02958b2312e1 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:01.849291 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 694/1387] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:01 2022] POST /identity/v3/users => generated 283 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:01.853127 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5285c6-0299-4984-b0e0-9f3da3e34dc9 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:01.860851 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b9fa5c-f4e6-425a-bab1-e7df34bb2bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:01.862761 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9fa5c-f4e6-425a-bab1-e7df34bb2bc5 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:01.863149 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9fa5c-f4e6-425a-bab1-e7df34bb2bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:01.864392 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9fa5c-f4e6-425a-bab1-e7df34bb2bc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:01.865796 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b9fa5c-f4e6-425a-bab1-e7df34bb2bc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:01.870831 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5285c6-0299-4984-b0e0-9f3da3e34dc9 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:01.878870 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b9fa5c-f4e6-425a-bab1-e7df34bb2bc5 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:01.879018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855ef79-addf-4ba1-98f5-fecb95d576ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:01.880091 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855ef79-addf-4ba1-98f5-fecb95d576ba admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:01.881073 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855ef79-addf-4ba1-98f5-fecb95d576ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:01.881546 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855ef79-addf-4ba1-98f5-fecb95d576ba admin admin] PATH_INFO: `/v3/projects/e2391d1f58f24d188963a0eb7cf57776/users/a7340f905c44482d9cb7d291356563c8/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:01.926826 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855ef79-addf-4ba1-98f5-fecb95d576ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2391d1f58f24d188963a0eb7cf57776, user_id=a7340f905c44482d9cb7d291356563c8, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:01.930312 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855ef79-addf-4ba1-98f5-fecb95d576ba admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:02.041384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 695/1388] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:01 2022] PUT /identity/v3/projects/e2391d1f58f24d188963a0eb7cf57776/users/a7340f905c44482d9cb7d291356563c8/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:02.043302 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855ef79-addf-4ba1-98f5-fecb95d576ba admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:02.050588 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 694/1389] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:01 2022] POST /identity/v3/projects => generated 387 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:02.051621 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b9fa5c-f4e6-425a-bab1-e7df34bb2bc5 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:02.257755 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b9fa5c-f4e6-425a-bab1-e7df34bb2bc5 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:02.277984 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855ef79-addf-4ba1-98f5-fecb95d576ba admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:02.284988 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e48cc-0a75-43ac-8f57-0bbeff787aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:02.287654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e48cc-0a75-43ac-8f57-0bbeff787aad admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:02.287654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e48cc-0a75-43ac-8f57-0bbeff787aad admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:02.287654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00e48cc-0a75-43ac-8f57-0bbeff787aad admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:02.299292 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ee1ad-f65d-4a50-ae7a-841367419e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:02.300610 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00e48cc-0a75-43ac-8f57-0bbeff787aad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:02.300935 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ee1ad-f65d-4a50-ae7a-841367419e0f admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:02.301662 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ee1ad-f65d-4a50-ae7a-841367419e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:02.302098 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ee1ad-f65d-4a50-ae7a-841367419e0f admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:02.303053 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00e48cc-0a75-43ac-8f57-0bbeff787aad admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:02.304720 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ee1ad-f65d-4a50-ae7a-841367419e0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:02.306718 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ee1ad-f65d-4a50-ae7a-841367419e0f admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:02.329405 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 696/1390] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:02.329405 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ee1ad-f65d-4a50-ae7a-841367419e0f admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:02.346886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 695/1391] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:02.346886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e48cc-0a75-43ac-8f57-0bbeff787aad admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:02.353477 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ee1ad-f65d-4a50-ae7a-841367419e0f admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:02.369932 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66998c-54a4-4d50-b601-dfcf75a0c956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:02.370830 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66998c-54a4-4d50-b601-dfcf75a0c956 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:02.371335 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66998c-54a4-4d50-b601-dfcf75a0c956 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:02.371760 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e66998c-54a4-4d50-b601-dfcf75a0c956 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:02.372569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e66998c-54a4-4d50-b601-dfcf75a0c956 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:02.373600 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00e48cc-0a75-43ac-8f57-0bbeff787aad admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:02.374874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e66998c-54a4-4d50-b601-dfcf75a0c956 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:02.384463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efda3f4-4bf2-4d7d-b7f1-ebe10d528ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:02.385156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efda3f4-4bf2-4d7d-b7f1-ebe10d528ce9 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:02.385445 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efda3f4-4bf2-4d7d-b7f1-ebe10d528ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:02.385733 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efda3f4-4bf2-4d7d-b7f1-ebe10d528ce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:02.386394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efda3f4-4bf2-4d7d-b7f1-ebe10d528ce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:02.404264 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efda3f4-4bf2-4d7d-b7f1-ebe10d528ce9 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:02.425193 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 696/1392] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:02.428791 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efda3f4-4bf2-4d7d-b7f1-ebe10d528ce9 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:02.452218 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efda3f4-4bf2-4d7d-b7f1-ebe10d528ce9 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:02.463342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31d1ad-e5c5-4c5a-81b9-ebabc2d3a9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:02.465219 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31d1ad-e5c5-4c5a-81b9-ebabc2d3a9e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:02.465696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31d1ad-e5c5-4c5a-81b9-ebabc2d3a9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:02.468446 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31d1ad-e5c5-4c5a-81b9-ebabc2d3a9e9 admin admin] PATH_INFO: `/v3/projects/e2391d1f58f24d188963a0eb7cf57776/users/a7340f905c44482d9cb7d291356563c8/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:02.479760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31d1ad-e5c5-4c5a-81b9-ebabc2d3a9e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2391d1f58f24d188963a0eb7cf57776, user_id=a7340f905c44482d9cb7d291356563c8, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:02.484234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 697/1393] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:02 2022] POST /identity/v3/users => generated 294 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:02.486596 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31d1ad-e5c5-4c5a-81b9-ebabc2d3a9e9 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:02.492309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66998c-54a4-4d50-b601-dfcf75a0c956 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:02.517051 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 697/1394] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:02 2022] PUT /identity/v3/projects/e2391d1f58f24d188963a0eb7cf57776/users/a7340f905c44482d9cb7d291356563c8/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:02.518159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31d1ad-e5c5-4c5a-81b9-ebabc2d3a9e9 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:02.522362 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e66998c-54a4-4d50-b601-dfcf75a0c956 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:02.735347 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31d1ad-e5c5-4c5a-81b9-ebabc2d3a9e9 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:02.747872 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:02.748962 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53868e99-52fd-4f4e-a1aa-addeeccbb1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:02.749598 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53868e99-52fd-4f4e-a1aa-addeeccbb1b1 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:02.750706 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53868e99-52fd-4f4e-a1aa-addeeccbb1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:02.752077 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53868e99-52fd-4f4e-a1aa-addeeccbb1b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:02.752077 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53868e99-52fd-4f4e-a1aa-addeeccbb1b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:02.753793 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53868e99-52fd-4f4e-a1aa-addeeccbb1b1 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:02.772857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 698/1395] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:02.774627 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9bcef-64a7-4022-b4f2-224cdb7dd0f8 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:02.774922 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9bcef-64a7-4022-b4f2-224cdb7dd0f8 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:02.775200 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d9bcef-64a7-4022-b4f2-224cdb7dd0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:02.778211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59186862-6bdd-447e-8142-a6407cf0f667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:02.778912 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59186862-6bdd-447e-8142-a6407cf0f667 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:02.780626 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59186862-6bdd-447e-8142-a6407cf0f667 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:02.780626 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59186862-6bdd-447e-8142-a6407cf0f667 admin admin] PATH_INFO: `/v3/projects/54aede97981945b297ac091584f01e9c/users/39c05f642d1f4ff6a77d3cff6e5527d7/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:02.805492 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59186862-6bdd-447e-8142-a6407cf0f667 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54aede97981945b297ac091584f01e9c, user_id=39c05f642d1f4ff6a77d3cff6e5527d7, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:02.808280 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59186862-6bdd-447e-8142-a6407cf0f667 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:02.811716 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-c8d9bcef-64a7-4022-b4f2-224cdb7dd0f8 None None] MFA Rules not processed for user `a7340f905c44482d9cb7d291356563c8`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:02.814257 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c8d9bcef-64a7-4022-b4f2-224cdb7dd0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:02.946856 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 699/1396] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:02 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:02.948366 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9263442-f81c-40e1-86e6-07eb18ee36ea None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:02.948777 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9263442-f81c-40e1-86e6-07eb18ee36ea None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:02.949058 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9263442-f81c-40e1-86e6-07eb18ee36ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:02.964610 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 698/1397] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:02 2022] PUT /identity/v3/projects/54aede97981945b297ac091584f01e9c/users/39c05f642d1f4ff6a77d3cff6e5527d7/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:02.965613 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59186862-6bdd-447e-8142-a6407cf0f667 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:03.008003 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-e9263442-f81c-40e1-86e6-07eb18ee36ea None None] MFA Rules not processed for user `ec2ca63a9f4c4c18bde25b1e4eab93ab`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:03.009115 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-e9263442-f81c-40e1-86e6-07eb18ee36ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:03.122496 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59186862-6bdd-447e-8142-a6407cf0f667 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:03.127647 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 700/1398] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:03.129208 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9263442-f81c-40e1-86e6-07eb18ee36ea None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:03.131415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-e9263442-f81c-40e1-86e6-07eb18ee36ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:03.144615 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1bd6d7-58d2-4f6e-a3ab-495506a49a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:03.145476 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bd6d7-58d2-4f6e-a3ab-495506a49a6e admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:03.145874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bd6d7-58d2-4f6e-a3ab-495506a49a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:03.147389 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1bd6d7-58d2-4f6e-a3ab-495506a49a6e admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:03.148308 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1bd6d7-58d2-4f6e-a3ab-495506a49a6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:03.150422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1bd6d7-58d2-4f6e-a3ab-495506a49a6e admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:03.203353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 699/1399] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:03.205337 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26356a5f-11dd-4ca8-83eb-04c33b5d8fef None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:03.206092 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26356a5f-11dd-4ca8-83eb-04c33b5d8fef None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:03.206881 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26356a5f-11dd-4ca8-83eb-04c33b5d8fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:03.296570 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9263442-f81c-40e1-86e6-07eb18ee36ea None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:03.306303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c29f3b-d67e-4de4-b323-d59c6d6673c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:03.317579 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c29f3b-d67e-4de4-b323-d59c6d6673c2 service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:03.318783 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c29f3b-d67e-4de4-b323-d59c6d6673c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:03.323200 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c29f3b-d67e-4de4-b323-d59c6d6673c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:03.330002 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-26356a5f-11dd-4ca8-83eb-04c33b5d8fef None None] MFA Rules not processed for user `39c05f642d1f4ff6a77d3cff6e5527d7`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:03.334239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-26356a5f-11dd-4ca8-83eb-04c33b5d8fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:03.496666 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 700/1400] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:03.497813 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26356a5f-11dd-4ca8-83eb-04c33b5d8fef None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:03.513615 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26356a5f-11dd-4ca8-83eb-04c33b5d8fef None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:03.519583 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96be23-97dc-444e-a7a0-2ba795984460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:03.520335 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96be23-97dc-444e-a7a0-2ba795984460 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:03.520666 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96be23-97dc-444e-a7a0-2ba795984460 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:03.521267 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96be23-97dc-444e-a7a0-2ba795984460 admin admin] PATH_INFO: `/v3/projects/5c336106a0134ac1b0085e6c05827482/users/93c9d09c49cd4b08825815e281c83f2e/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:03.547055 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf96be23-97dc-444e-a7a0-2ba795984460 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c336106a0134ac1b0085e6c05827482, user_id=93c9d09c49cd4b08825815e281c83f2e, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:03.568906 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf96be23-97dc-444e-a7a0-2ba795984460 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:03.654918 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c29f3b-d67e-4de4-b323-d59c6d6673c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:03.656833 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c29f3b-d67e-4de4-b323-d59c6d6673c2 service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:03.664712 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 701/1401] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:03 2022] PUT /identity/v3/projects/5c336106a0134ac1b0085e6c05827482/users/93c9d09c49cd4b08825815e281c83f2e/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:03.666241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96be23-97dc-444e-a7a0-2ba795984460 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:03.693935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 701/1402] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:03 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 565 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:03.695528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36c719-3f21-43de-b162-866a4c03a5ec None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:03.697952 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36c719-3f21-43de-b162-866a4c03a5ec None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:03.699432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36c719-3f21-43de-b162-866a4c03a5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:03.793528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-1d36c719-3f21-43de-b162-866a4c03a5ec None None] MFA Rules not processed for user `93c9d09c49cd4b08825815e281c83f2e`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:03.797720 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-1d36c719-3f21-43de-b162-866a4c03a5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:03.891182 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96be23-97dc-444e-a7a0-2ba795984460 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:03.899012 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb73733-1526-4b42-877b-17519ed68462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:03.899892 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb73733-1526-4b42-877b-17519ed68462 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:03.900102 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb73733-1526-4b42-877b-17519ed68462 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:03.900476 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb73733-1526-4b42-877b-17519ed68462 admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:03.901182 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb73733-1526-4b42-877b-17519ed68462 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:03.907649 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb73733-1526-4b42-877b-17519ed68462 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:04.039267 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 702/1403] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:03 2022] POST /identity/v3/projects => generated 363 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:04.043767 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb73733-1526-4b42-877b-17519ed68462 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:04.065687 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 702/1404] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:04.066866 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36c719-3f21-43de-b162-866a4c03a5ec None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:04.188216 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36c719-3f21-43de-b162-866a4c03a5ec None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:04.209242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb73733-1526-4b42-877b-17519ed68462 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:04.223798 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d35fa0-8501-444f-9d9f-24d264b419e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:04.225210 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d35fa0-8501-444f-9d9f-24d264b419e0 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:04.225640 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d35fa0-8501-444f-9d9f-24d264b419e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:04.234566 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d35fa0-8501-444f-9d9f-24d264b419e0 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:04.235341 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d35fa0-8501-444f-9d9f-24d264b419e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:04.238502 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d35fa0-8501-444f-9d9f-24d264b419e0 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:04.328464 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126423ab-d7e9-4249-834c-223bc7aaab13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:04.329110 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126423ab-d7e9-4249-834c-223bc7aaab13 service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:04.329410 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126423ab-d7e9-4249-834c-223bc7aaab13 service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:04.329665 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126423ab-d7e9-4249-834c-223bc7aaab13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:04.384440 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 703/1405] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:04 2022] POST /identity/v3/users => generated 288 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:04.385300 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d35fa0-8501-444f-9d9f-24d264b419e0 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:04.404106 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:04.410764 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d35fa0-8501-444f-9d9f-24d264b419e0 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:04.426355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c9bd1b-7929-4dcc-91b6-9946d22c1146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:04.428879 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9bd1b-7929-4dcc-91b6-9946d22c1146 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:04.429900 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9bd1b-7929-4dcc-91b6-9946d22c1146 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:04.430259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9bd1b-7929-4dcc-91b6-9946d22c1146 admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:04.436888 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c9bd1b-7929-4dcc-91b6-9946d22c1146 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:04.439793 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c9bd1b-7929-4dcc-91b6-9946d22c1146 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:04.490108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126423ab-d7e9-4249-834c-223bc7aaab13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:04.492077 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126423ab-d7e9-4249-834c-223bc7aaab13 service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:04.508026 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 703/1406] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:04 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:04.508898 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126423ab-d7e9-4249-834c-223bc7aaab13 service cinder] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:04.518731 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:04.537559 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126423ab-d7e9-4249-834c-223bc7aaab13 service cinder] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:04.547894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580db761-8ebc-4f7a-80ef-dc90f6254f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:04.548816 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580db761-8ebc-4f7a-80ef-dc90f6254f2e admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:04.549283 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580db761-8ebc-4f7a-80ef-dc90f6254f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:04.549697 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580db761-8ebc-4f7a-80ef-dc90f6254f2e admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:04.550533 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580db761-8ebc-4f7a-80ef-dc90f6254f2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:04.553058 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580db761-8ebc-4f7a-80ef-dc90f6254f2e admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:04.574230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 704/1407] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:04.577138 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d93270-9e05-48f7-aef7-de34f889b43d None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:04.579201 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d93270-9e05-48f7-aef7-de34f889b43d None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:04.579647 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d93270-9e05-48f7-aef7-de34f889b43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:04.624833 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 704/1408] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:04 2022] POST /identity/v3/projects => generated 387 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:04.624999 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-f4d93270-9e05-48f7-aef7-de34f889b43d None None] MFA Rules not processed for user `57ee3a13e33549c9a90ebde353fcd6f2`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:04.626256 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-f4d93270-9e05-48f7-aef7-de34f889b43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:04.629154 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c9bd1b-7929-4dcc-91b6-9946d22c1146 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:04.782197 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 705/1409] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:04 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:04.801839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d93270-9e05-48f7-aef7-de34f889b43d None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:04.882774 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c9bd1b-7929-4dcc-91b6-9946d22c1146 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:04.896511 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ca231-1866-4c00-abe9-56b54a6658c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:04.896911 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ca231-1866-4c00-abe9-56b54a6658c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:04.897329 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ca231-1866-4c00-abe9-56b54a6658c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:04.897742 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ca231-1866-4c00-abe9-56b54a6658c4 admin admin] PATH_INFO: `/v3/projects/c80adecb90e44f7886c97b41b26f7f8f/users/20b6e2199ac548f29828047a40b7e1a6/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:04.958611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ca231-1866-4c00-abe9-56b54a6658c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c80adecb90e44f7886c97b41b26f7f8f, user_id=20b6e2199ac548f29828047a40b7e1a6, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:04.962250 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ca231-1866-4c00-abe9-56b54a6658c4 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:04.978313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d93270-9e05-48f7-aef7-de34f889b43d None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:04.999809 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9dc644-30c1-4021-814d-012ca3b2ce41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:05.001009 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9dc644-30c1-4021-814d-012ca3b2ce41 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:05.002399 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9dc644-30c1-4021-814d-012ca3b2ce41 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:05.002728 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9dc644-30c1-4021-814d-012ca3b2ce41 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:05.004947 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9dc644-30c1-4021-814d-012ca3b2ce41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:05.013735 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9dc644-30c1-4021-814d-012ca3b2ce41 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:05.072692 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 705/1410] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:04 2022] PUT /identity/v3/projects/c80adecb90e44f7886c97b41b26f7f8f/users/20b6e2199ac548f29828047a40b7e1a6/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:05.077880 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ca231-1866-4c00-abe9-56b54a6658c4 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:05.164007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 706/1411] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:04 2022] POST /identity/v3/users => generated 300 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:05.168062 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9dc644-30c1-4021-814d-012ca3b2ce41 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:05.201697 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9dc644-30c1-4021-814d-012ca3b2ce41 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:05.341010 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ca231-1866-4c00-abe9-56b54a6658c4 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:05.365942 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43e8e0-9ebd-4e87-901f-cc22502d709d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:05.366578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43e8e0-9ebd-4e87-901f-cc22502d709d admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:05.366861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43e8e0-9ebd-4e87-901f-cc22502d709d admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:05.379805 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f43e8e0-9ebd-4e87-901f-cc22502d709d admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:05.379805 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43e8e0-9ebd-4e87-901f-cc22502d709d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:05.386960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f43e8e0-9ebd-4e87-901f-cc22502d709d admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:05.435824 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 706/1412] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:05.440116 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43e8e0-9ebd-4e87-901f-cc22502d709d admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:05.463130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a8c95-e2a1-49ab-82c1-b2d31d32f58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:05.468256 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a8c95-e2a1-49ab-82c1-b2d31d32f58b admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:05.468599 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a8c95-e2a1-49ab-82c1-b2d31d32f58b admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:05.468879 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a8c95-e2a1-49ab-82c1-b2d31d32f58b admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:05.470549 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a8c95-e2a1-49ab-82c1-b2d31d32f58b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:05.471469 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f43e8e0-9ebd-4e87-901f-cc22502d709d admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:05.477576 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a8c95-e2a1-49ab-82c1-b2d31d32f58b admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:05.482174 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886aec9e-f4f1-41e7-9462-b1b66ac935e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:05.483383 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886aec9e-f4f1-41e7-9462-b1b66ac935e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:05.483914 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886aec9e-f4f1-41e7-9462-b1b66ac935e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:05.484408 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886aec9e-f4f1-41e7-9462-b1b66ac935e7 admin admin] PATH_INFO: `/v3/projects/c80adecb90e44f7886c97b41b26f7f8f/users/20b6e2199ac548f29828047a40b7e1a6/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:05.486748 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886aec9e-f4f1-41e7-9462-b1b66ac935e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c80adecb90e44f7886c97b41b26f7f8f, user_id=20b6e2199ac548f29828047a40b7e1a6, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:05.490721 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886aec9e-f4f1-41e7-9462-b1b66ac935e7 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:05.493856 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 707/1413] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:05.500063 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a8c95-e2a1-49ab-82c1-b2d31d32f58b admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:05.514581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 707/1414] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:05 2022] PUT /identity/v3/projects/c80adecb90e44f7886c97b41b26f7f8f/users/20b6e2199ac548f29828047a40b7e1a6/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:51:05.514756 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a8c95-e2a1-49ab-82c1-b2d31d32f58b admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:05.517144 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660b1a7-1d64-4244-bc4f-e7e84c13a0b6 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:05.517617 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660b1a7-1d64-4244-bc4f-e7e84c13a0b6 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:05.518068 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660b1a7-1d64-4244-bc4f-e7e84c13a0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:05.583129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-e660b1a7-1d64-4244-bc4f-e7e84c13a0b6 None None] MFA Rules not processed for user `20b6e2199ac548f29828047a40b7e1a6`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:05.586211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-e660b1a7-1d64-4244-bc4f-e7e84c13a0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:05.587838 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:05.609462 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61afd84e-da4e-46c0-9724-97066b0c74b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:05.610146 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61afd84e-da4e-46c0-9724-97066b0c74b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:05.610433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61afd84e-da4e-46c0-9724-97066b0c74b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:05.610713 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61afd84e-da4e-46c0-9724-97066b0c74b0 admin admin] PATH_INFO: `/v3/projects/4de3fe0177254025a2aeb933a9df77e2/users/75cc2c60ed2449adaa058b9f8688e254/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:05.637806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61afd84e-da4e-46c0-9724-97066b0c74b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4de3fe0177254025a2aeb933a9df77e2, user_id=75cc2c60ed2449adaa058b9f8688e254, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:05.642382 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61afd84e-da4e-46c0-9724-97066b0c74b0 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:05.698418 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:05.716752 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 708/1415] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:05 2022] PUT /identity/v3/projects/4de3fe0177254025a2aeb933a9df77e2/users/75cc2c60ed2449adaa058b9f8688e254/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:05.720892 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61afd84e-da4e-46c0-9724-97066b0c74b0 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:05.757327 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 708/1416] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:05 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:51:05.767057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660b1a7-1d64-4244-bc4f-e7e84c13a0b6 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:05.822291 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61afd84e-da4e-46c0-9724-97066b0c74b0 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:05.834276 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc978c6-1fd2-446a-b861-2a25d582f10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:05.834834 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc978c6-1fd2-446a-b861-2a25d582f10e admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:05.835121 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc978c6-1fd2-446a-b861-2a25d582f10e admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:05.835400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc978c6-1fd2-446a-b861-2a25d582f10e admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:05.836579 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc978c6-1fd2-446a-b861-2a25d582f10e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:05.838189 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc978c6-1fd2-446a-b861-2a25d582f10e admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:05.858387 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 709/1417] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:05.862873 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc978c6-1fd2-446a-b861-2a25d582f10e admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:05.875063 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc978c6-1fd2-446a-b861-2a25d582f10e admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:05.879622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660b1a7-1d64-4244-bc4f-e7e84c13a0b6 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:05.882162 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d2350-4400-47c9-9adb-a89769207246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:05.883244 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d2350-4400-47c9-9adb-a89769207246 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:05.883722 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d2350-4400-47c9-9adb-a89769207246 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:05.884365 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d2350-4400-47c9-9adb-a89769207246 admin admin] PATH_INFO: `/v3/projects/4de3fe0177254025a2aeb933a9df77e2/users/75cc2c60ed2449adaa058b9f8688e254/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:05.887319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d2350-4400-47c9-9adb-a89769207246 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4de3fe0177254025a2aeb933a9df77e2, user_id=75cc2c60ed2449adaa058b9f8688e254, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:05.890526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d2350-4400-47c9-9adb-a89769207246 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:05.906339 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 710/1418] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:05 2022] PUT /identity/v3/projects/4de3fe0177254025a2aeb933a9df77e2/users/75cc2c60ed2449adaa058b9f8688e254/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:05.916703 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec264ed-e105-4ea2-bde9-812255c4bc63 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:05.917254 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec264ed-e105-4ea2-bde9-812255c4bc63 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:05.917845 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec264ed-e105-4ea2-bde9-812255c4bc63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:05.947279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3581166-eb6c-4aba-ba23-fbd641e53232 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:05.947915 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3581166-eb6c-4aba-ba23-fbd641e53232 service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:05.948269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3581166-eb6c-4aba-ba23-fbd641e53232 service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:05.948934 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3581166-eb6c-4aba-ba23-fbd641e53232 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:05.952021 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-6ec264ed-e105-4ea2-bde9-812255c4bc63 None None] MFA Rules not processed for user `75cc2c60ed2449adaa058b9f8688e254`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:05.953438 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-6ec264ed-e105-4ea2-bde9-812255c4bc63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:06.059248 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 711/1419] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:05 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:51:06.074519 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec264ed-e105-4ea2-bde9-812255c4bc63 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:06.074942 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3581166-eb6c-4aba-ba23-fbd641e53232 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:06.078174 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-6ec264ed-e105-4ea2-bde9-812255c4bc63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:06.089084 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3581166-eb6c-4aba-ba23-fbd641e53232 service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:06.110446 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 709/1420] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:05 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:06.155289 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec264ed-e105-4ea2-bde9-812255c4bc63 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:06.160509 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09efad6-9e6a-4394-9f32-2fcdb48981fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:06.161482 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09efad6-9e6a-4394-9f32-2fcdb48981fc service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:06.161980 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09efad6-9e6a-4394-9f32-2fcdb48981fc service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:06.162440 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09efad6-9e6a-4394-9f32-2fcdb48981fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:06.182196 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09efad6-9e6a-4394-9f32-2fcdb48981fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:06.183668 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09efad6-9e6a-4394-9f32-2fcdb48981fc service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:06.215367 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 712/1421] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:06 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:51:06.267611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:06.360732 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3581166-eb6c-4aba-ba23-fbd641e53232 service cinder] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:06.368107 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:06.383995 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3581166-eb6c-4aba-ba23-fbd641e53232 service cinder] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:06.505865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09efad6-9e6a-4394-9f32-2fcdb48981fc service cinder] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:06.515188 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09efad6-9e6a-4394-9f32-2fcdb48981fc service cinder] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:06.529442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7b464-7d71-476f-8e61-56003d96f12d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:06.530149 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7b464-7d71-476f-8e61-56003d96f12d service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:06.535165 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7b464-7d71-476f-8e61-56003d96f12d service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:06.535575 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7b464-7d71-476f-8e61-56003d96f12d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:06.618038 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:06.652518 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd19c49-1b7f-4a3b-850f-1b8b37fec53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:06.653627 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd19c49-1b7f-4a3b-850f-1b8b37fec53c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:06.653627 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd19c49-1b7f-4a3b-850f-1b8b37fec53c admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:06.653811 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd19c49-1b7f-4a3b-850f-1b8b37fec53c admin admin] PATH_INFO: `/v3/users/93c9d09c49cd4b08825815e281c83f2e` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:06.656573 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd19c49-1b7f-4a3b-850f-1b8b37fec53c admin admin] RBAC: Authorizing `identity:delete_user(user_id=93c9d09c49cd4b08825815e281c83f2e)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:06.662333 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd19c49-1b7f-4a3b-850f-1b8b37fec53c admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:06.712775 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7b464-7d71-476f-8e61-56003d96f12d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:06.713781 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7b464-7d71-476f-8e61-56003d96f12d service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:06.740228 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 713/1422] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:06.861080 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:06.949735 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2bd19c49-1b7f-4a3b-850f-1b8b37fec53c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93c9d09c49cd4b08825815e281c83f2e'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:06.950398 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2bd19c49-1b7f-4a3b-850f-1b8b37fec53c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93c9d09c49cd4b08825815e281c83f2e'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:07.060511 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2bd19c49-1b7f-4a3b-850f-1b8b37fec53c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93c9d09c49cd4b08825815e281c83f2e'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:07.088928 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2bd19c49-1b7f-4a3b-850f-1b8b37fec53c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93c9d09c49cd4b08825815e281c83f2e'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:07.090001 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2bd19c49-1b7f-4a3b-850f-1b8b37fec53c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93c9d09c49cd4b08825815e281c83f2e'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:07.152377 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 710/1423] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:51:06 2022] DELETE /identity/v3/users/93c9d09c49cd4b08825815e281c83f2e => generated 0 bytes in 792 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:07.159030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f9826-1cfb-49da-bd38-edc69b60ae70 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:07.159502 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f9826-1cfb-49da-bd38-edc69b60ae70 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:07.159879 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f9826-1cfb-49da-bd38-edc69b60ae70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:07.187887 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-9e4f9826-1cfb-49da-bd38-edc69b60ae70 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:07.189346 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-9e4f9826-1cfb-49da-bd38-edc69b60ae70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:07.255867 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 714/1424] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:51:07.267985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd19c49-1b7f-4a3b-850f-1b8b37fec53c admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:07.271690 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-2bd19c49-1b7f-4a3b-850f-1b8b37fec53c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:07.351912 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd19c49-1b7f-4a3b-850f-1b8b37fec53c admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:07.356205 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bebe14-48fa-4d74-9ed7-93080beccc58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:07.356947 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bebe14-48fa-4d74-9ed7-93080beccc58 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:07.360603 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bebe14-48fa-4d74-9ed7-93080beccc58 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:07.360603 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bebe14-48fa-4d74-9ed7-93080beccc58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:07.368194 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bebe14-48fa-4d74-9ed7-93080beccc58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:07.369765 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bebe14-48fa-4d74-9ed7-93080beccc58 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:07.373644 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc35059-03a5-4f99-9eb6-25f56738df93 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:07.373939 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc35059-03a5-4f99-9eb6-25f56738df93 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:07.374229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc35059-03a5-4f99-9eb6-25f56738df93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:07.380649 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 711/1425] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:07.393155 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:07.547207 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-bbc35059-03a5-4f99-9eb6-25f56738df93 None None] MFA Rules not processed for user `cecbc19355ef4570b2b05bba876a3a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:07.559037 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-bbc35059-03a5-4f99-9eb6-25f56738df93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:07.720438 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 715/1426] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Oct 17 03:51:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:08.203355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001c5a9-07e8-4c25-bb7c-821b9817aae3 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:08.203646 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001c5a9-07e8-4c25-bb7c-821b9817aae3 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:08.203957 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001c5a9-07e8-4c25-bb7c-821b9817aae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:08.234095 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-6001c5a9-07e8-4c25-bb7c-821b9817aae3 None None] MFA Rules not processed for user `414707c545f641d3935720356b326672`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:08.235334 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6001c5a9-07e8-4c25-bb7c-821b9817aae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:08.261649 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc35059-03a5-4f99-9eb6-25f56738df93 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:08.263227 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-bbc35059-03a5-4f99-9eb6-25f56738df93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:08.289651 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:08.301410 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 712/1427] 10.222.0.33 () {58 vars in 1015 bytes} [Mon Oct 17 03:51:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:08.340258 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6001c5a9-07e8-4c25-bb7c-821b9817aae3 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:08.341639 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6001c5a9-07e8-4c25-bb7c-821b9817aae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:08.358870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6001c5a9-07e8-4c25-bb7c-821b9817aae3 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:08.379565 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571834f-d7bd-4b2e-9fd2-0d8a32659531 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:08.379565 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571834f-d7bd-4b2e-9fd2-0d8a32659531 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:08.379565 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571834f-d7bd-4b2e-9fd2-0d8a32659531 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:08.379565 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571834f-d7bd-4b2e-9fd2-0d8a32659531 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:08.386415 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571834f-d7bd-4b2e-9fd2-0d8a32659531 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:08.390710 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571834f-d7bd-4b2e-9fd2-0d8a32659531 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:08.402526 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 713/1428] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:08.479767 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc35059-03a5-4f99-9eb6-25f56738df93 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:08.485663 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452a4d0-e1b9-4dd1-9776-97215ea82da5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f64aaf4d408462a88963973d42abfb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:08.486651 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452a4d0-e1b9-4dd1-9776-97215ea82da5 service swift] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:08.487169 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452a4d0-e1b9-4dd1-9776-97215ea82da5 service swift] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:08.487662 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452a4d0-e1b9-4dd1-9776-97215ea82da5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:08.498839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c452a4d0-e1b9-4dd1-9776-97215ea82da5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:08.501127 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c452a4d0-e1b9-4dd1-9776-97215ea82da5 service swift] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:08.532043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 716/1429] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Oct 17 03:51:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 17 03:51:08.700058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571834f-d7bd-4b2e-9fd2-0d8a32659531 service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:08.701777 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-1571834f-d7bd-4b2e-9fd2-0d8a32659531 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:08.746048 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571834f-d7bd-4b2e-9fd2-0d8a32659531 service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:08.777855 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85685ace-5c10-4436-9ab6-25da0b69a16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:08.779241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85685ace-5c10-4436-9ab6-25da0b69a16e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:08.780386 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85685ace-5c10-4436-9ab6-25da0b69a16e admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:08.781178 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85685ace-5c10-4436-9ab6-25da0b69a16e admin admin] PATH_INFO: `/v3/projects/5c336106a0134ac1b0085e6c05827482` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:08.782641 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85685ace-5c10-4436-9ab6-25da0b69a16e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c336106a0134ac1b0085e6c05827482)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:08.785077 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85685ace-5c10-4436-9ab6-25da0b69a16e admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:09.051000 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-85685ace-5c10-4436-9ab6-25da0b69a16e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c336106a0134ac1b0085e6c05827482', 'request_id': 'req-85685ace-5c10-4436-9ab6-25da0b69a16e'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:09.231159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-85685ace-5c10-4436-9ab6-25da0b69a16e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c336106a0134ac1b0085e6c05827482', 'request_id': 'req-85685ace-5c10-4436-9ab6-25da0b69a16e'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:09.249552 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-85685ace-5c10-4436-9ab6-25da0b69a16e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c336106a0134ac1b0085e6c05827482', 'request_id': 'req-85685ace-5c10-4436-9ab6-25da0b69a16e'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:09.289483 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-85685ace-5c10-4436-9ab6-25da0b69a16e admin admin] The token cache is being invalidate because project 5c336106a0134ac1b0085e6c05827482 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:09.290421 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-85685ace-5c10-4436-9ab6-25da0b69a16e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:09.297705 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-85685ace-5c10-4436-9ab6-25da0b69a16e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:09.303051 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 714/1430] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:08 2022] DELETE /identity/v3/projects/5c336106a0134ac1b0085e6c05827482 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:09.307634 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452a4d0-e1b9-4dd1-9776-97215ea82da5 service swift] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:09.348469 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452a4d0-e1b9-4dd1-9776-97215ea82da5 service swift] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:09.374217 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:09.602498 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e141a-5836-49ff-bce0-fbe3bf33c26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:09.603422 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e141a-5836-49ff-bce0-fbe3bf33c26d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:09.607236 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e141a-5836-49ff-bce0-fbe3bf33c26d admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:09.607767 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e141a-5836-49ff-bce0-fbe3bf33c26d admin admin] PATH_INFO: `/v3/users/75cc2c60ed2449adaa058b9f8688e254` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:09.625308 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e141a-5836-49ff-bce0-fbe3bf33c26d admin admin] RBAC: Authorizing `identity:delete_user(user_id=75cc2c60ed2449adaa058b9f8688e254)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:09.632852 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e141a-5836-49ff-bce0-fbe3bf33c26d admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:09.933317 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-915e141a-5836-49ff-bce0-fbe3bf33c26d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75cc2c60ed2449adaa058b9f8688e254'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:10.199805 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-915e141a-5836-49ff-bce0-fbe3bf33c26d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75cc2c60ed2449adaa058b9f8688e254'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:10.243962 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-915e141a-5836-49ff-bce0-fbe3bf33c26d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75cc2c60ed2449adaa058b9f8688e254'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:10.246059 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-915e141a-5836-49ff-bce0-fbe3bf33c26d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75cc2c60ed2449adaa058b9f8688e254'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:10.247652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-915e141a-5836-49ff-bce0-fbe3bf33c26d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75cc2c60ed2449adaa058b9f8688e254'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:10.306536 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 717/1431] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:51:09 2022] DELETE /identity/v3/users/75cc2c60ed2449adaa058b9f8688e254 => generated 0 bytes in 999 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:10.986685 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85685ace-5c10-4436-9ab6-25da0b69a16e admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:10.988567 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-85685ace-5c10-4436-9ab6-25da0b69a16e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:11.003507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85685ace-5c10-4436-9ab6-25da0b69a16e admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:11.038273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:11.082810 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309f5b4-443e-41c6-bca0-e5610dcbceca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:11.084263 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309f5b4-443e-41c6-bca0-e5610dcbceca admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:11.084624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309f5b4-443e-41c6-bca0-e5610dcbceca admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:11.084985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309f5b4-443e-41c6-bca0-e5610dcbceca admin admin] PATH_INFO: `/v3/users/5bcf4a4b0ab14a118be938a5ee980e08` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:11.086294 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1309f5b4-443e-41c6-bca0-e5610dcbceca admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bcf4a4b0ab14a118be938a5ee980e08)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:11.088697 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1309f5b4-443e-41c6-bca0-e5610dcbceca admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:11.306019 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-1309f5b4-443e-41c6-bca0-e5610dcbceca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bcf4a4b0ab14a118be938a5ee980e08'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:11.306973 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-1309f5b4-443e-41c6-bca0-e5610dcbceca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bcf4a4b0ab14a118be938a5ee980e08'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:11.387949 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-1309f5b4-443e-41c6-bca0-e5610dcbceca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bcf4a4b0ab14a118be938a5ee980e08'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:11.403388 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-1309f5b4-443e-41c6-bca0-e5610dcbceca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bcf4a4b0ab14a118be938a5ee980e08'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:11.404416 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-1309f5b4-443e-41c6-bca0-e5610dcbceca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bcf4a4b0ab14a118be938a5ee980e08'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:11.433221 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 715/1432] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:51:10 2022] DELETE /identity/v3/users/5bcf4a4b0ab14a118be938a5ee980e08 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:11.437633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1d788-3e20-4c42-94b4-48e0180c7be7 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:11.441968 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1d788-3e20-4c42-94b4-48e0180c7be7 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:11.441968 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1d788-3e20-4c42-94b4-48e0180c7be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:11.446049 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e753b3e-83a4-43ed-b612-e6086cde3d67 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:11.449008 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e753b3e-83a4-43ed-b612-e6086cde3d67 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:11.450209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e753b3e-83a4-43ed-b612-e6086cde3d67 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:11.455680 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 716/1433] 10.222.0.17 () {54 vars in 921 bytes} [Mon Oct 17 03:51:11 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:51:11.464803 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-ccc1d788-3e20-4c42-94b4-48e0180c7be7 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:11.466435 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-ccc1d788-3e20-4c42-94b4-48e0180c7be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:11.497303 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3a153-c026-471f-a283-b70d945d5757 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:11.497595 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3a153-c026-471f-a283-b70d945d5757 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:11.497877 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3a153-c026-471f-a283-b70d945d5757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:11.537787 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-9fe3a153-c026-471f-a283-b70d945d5757 None None] MFA Rules not processed for user `d961b751763a4e49b5d4ca4c795c5bf6`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:11.538821 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:11.540029 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-9fe3a153-c026-471f-a283-b70d945d5757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:11.562070 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 718/1434] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:11.572973 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1d788-3e20-4c42-94b4-48e0180c7be7 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:11.574459 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-ccc1d788-3e20-4c42-94b4-48e0180c7be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:11.657595 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 717/1435] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Oct 17 03:51:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:51:11.658505 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3a153-c026-471f-a283-b70d945d5757 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:11.659998 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-9fe3a153-c026-471f-a283-b70d945d5757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:11.660612 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1d788-3e20-4c42-94b4-48e0180c7be7 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:11.668661 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea66867b-b5f6-4cd6-ab01-d14346585ab2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:11.669295 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66867b-b5f6-4cd6-ab01-d14346585ab2 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:11.669800 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66867b-b5f6-4cd6-ab01-d14346585ab2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:11.671264 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea66867b-b5f6-4cd6-ab01-d14346585ab2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:11.686081 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3a153-c026-471f-a283-b70d945d5757 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:11.690551 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea66867b-b5f6-4cd6-ab01-d14346585ab2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:11.691347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a8db7-a5d3-428d-9005-db899d3892b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:11.692866 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a8db7-a5d3-428d-9005-db899d3892b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:11.693310 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a8db7-a5d3-428d-9005-db899d3892b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:11.693745 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a8db7-a5d3-428d-9005-db899d3892b5 admin admin] PATH_INFO: `/v3/projects/4de3fe0177254025a2aeb933a9df77e2` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:11.693936 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea66867b-b5f6-4cd6-ab01-d14346585ab2 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:11.695206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a8db7-a5d3-428d-9005-db899d3892b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4de3fe0177254025a2aeb933a9df77e2)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:11.697323 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a8db7-a5d3-428d-9005-db899d3892b5 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:11.712836 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 719/1436] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:51:11.713788 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea66867b-b5f6-4cd6-ab01-d14346585ab2 service neutron] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:11.715252 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-ea66867b-b5f6-4cd6-ab01-d14346585ab2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:11.736069 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea66867b-b5f6-4cd6-ab01-d14346585ab2 service neutron] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:11.756872 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb9dcd-8d01-4680-8e33-d5562aaabd36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:11.757769 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb9dcd-8d01-4680-8e33-d5562aaabd36 service nova] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:11.758055 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb9dcd-8d01-4680-8e33-d5562aaabd36 service nova] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:11.758342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb9dcd-8d01-4680-8e33-d5562aaabd36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:11.779166 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb9dcd-8d01-4680-8e33-d5562aaabd36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:11.781023 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb9dcd-8d01-4680-8e33-d5562aaabd36 service nova] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:11.806104 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 720/1437] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:11.878135 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:11.888291 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-af4a8db7-a5d3-428d-9005-db899d3892b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4de3fe0177254025a2aeb933a9df77e2', 'request_id': 'req-af4a8db7-a5d3-428d-9005-db899d3892b5'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:11.998037 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-af4a8db7-a5d3-428d-9005-db899d3892b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4de3fe0177254025a2aeb933a9df77e2', 'request_id': 'req-af4a8db7-a5d3-428d-9005-db899d3892b5'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:12.005371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-af4a8db7-a5d3-428d-9005-db899d3892b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4de3fe0177254025a2aeb933a9df77e2', 'request_id': 'req-af4a8db7-a5d3-428d-9005-db899d3892b5'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:12.017331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-af4a8db7-a5d3-428d-9005-db899d3892b5 admin admin] The token cache is being invalidate because project 4de3fe0177254025a2aeb933a9df77e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:12.017687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-af4a8db7-a5d3-428d-9005-db899d3892b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:12.018593 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-af4a8db7-a5d3-428d-9005-db899d3892b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:12.020483 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 718/1438] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:11 2022] DELETE /identity/v3/projects/4de3fe0177254025a2aeb933a9df77e2 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:12.026503 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac79da-e6e9-4ca0-b524-84bcf420d834 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:12.026503 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac79da-e6e9-4ca0-b524-84bcf420d834 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:12.026503 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac79da-e6e9-4ca0-b524-84bcf420d834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:12.050750 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-97ac79da-e6e9-4ca0-b524-84bcf420d834 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:12.052334 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-97ac79da-e6e9-4ca0-b524-84bcf420d834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:12.122077 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 721/1439] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:12.127022 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295f1c4-e7a9-4c72-b68b-7122a548c098 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:12.127311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295f1c4-e7a9-4c72-b68b-7122a548c098 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:12.127639 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295f1c4-e7a9-4c72-b68b-7122a548c098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:12.156852 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-c295f1c4-e7a9-4c72-b68b-7122a548c098 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:12.158042 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-c295f1c4-e7a9-4c72-b68b-7122a548c098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:12.180555 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 719/1440] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:12.184890 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac79da-e6e9-4ca0-b524-84bcf420d834 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:12.199033 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac79da-e6e9-4ca0-b524-84bcf420d834 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:12.205276 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d5aaf-baa2-433e-b4dd-d6420ad32276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:12.206182 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d5aaf-baa2-433e-b4dd-d6420ad32276 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:12.206809 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d5aaf-baa2-433e-b4dd-d6420ad32276 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:12.209136 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d5aaf-baa2-433e-b4dd-d6420ad32276 admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:12.210143 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d5aaf-baa2-433e-b4dd-d6420ad32276 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:51:12.210612 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d5aaf-baa2-433e-b4dd-d6420ad32276 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:12.212400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d5aaf-baa2-433e-b4dd-d6420ad32276 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:12.226397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 722/1441] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:51:12.230869 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295f1c4-e7a9-4c72-b68b-7122a548c098 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:12.243705 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295f1c4-e7a9-4c72-b68b-7122a548c098 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:12.255065 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3177a1f9-bc9c-4231-bf03-e50457689180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:12.255948 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177a1f9-bc9c-4231-bf03-e50457689180 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:12.256069 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177a1f9-bc9c-4231-bf03-e50457689180 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:12.256254 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177a1f9-bc9c-4231-bf03-e50457689180 admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:12.256940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3177a1f9-bc9c-4231-bf03-e50457689180 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:12.258944 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3177a1f9-bc9c-4231-bf03-e50457689180 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:12.308156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d5aaf-baa2-433e-b4dd-d6420ad32276 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:12.310040 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-a40d5aaf-baa2-433e-b4dd-d6420ad32276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:12.325943 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d5aaf-baa2-433e-b4dd-d6420ad32276 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:12.330916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89166e9-c163-45ef-a176-fca6bfda9953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:12.331852 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89166e9-c163-45ef-a176-fca6bfda9953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:12.332144 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89166e9-c163-45ef-a176-fca6bfda9953 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:12.332532 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89166e9-c163-45ef-a176-fca6bfda9953 admin admin] PATH_INFO: `/v3/projects/3a5c79ca08614d7ba9cad0da6f32cccf` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:12.334252 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89166e9-c163-45ef-a176-fca6bfda9953 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a5c79ca08614d7ba9cad0da6f32cccf)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:12.336291 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89166e9-c163-45ef-a176-fca6bfda9953 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:12.419777 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 720/1442] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:12 2022] POST /identity/v3/projects => generated 389 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:12.423768 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3177a1f9-bc9c-4231-bf03-e50457689180 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:12.552020 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3177a1f9-bc9c-4231-bf03-e50457689180 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:12.552991 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-d89166e9-c163-45ef-a176-fca6bfda9953 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a5c79ca08614d7ba9cad0da6f32cccf', 'request_id': 'req-d89166e9-c163-45ef-a176-fca6bfda9953'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:12.637986 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-d89166e9-c163-45ef-a176-fca6bfda9953 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a5c79ca08614d7ba9cad0da6f32cccf', 'request_id': 'req-d89166e9-c163-45ef-a176-fca6bfda9953'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:12.640461 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:12.644686 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-d89166e9-c163-45ef-a176-fca6bfda9953 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a5c79ca08614d7ba9cad0da6f32cccf', 'request_id': 'req-d89166e9-c163-45ef-a176-fca6bfda9953'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:12.653633 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f5ce8-d9a9-45a5-b4ed-3824bb4cbc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:12.653633 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f5ce8-d9a9-45a5-b4ed-3824bb4cbc68 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:12.653633 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f5ce8-d9a9-45a5-b4ed-3824bb4cbc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:12.653633 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f5ce8-d9a9-45a5-b4ed-3824bb4cbc68 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:12.654241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f5ce8-d9a9-45a5-b4ed-3824bb4cbc68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:12.656168 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0f5ce8-d9a9-45a5-b4ed-3824bb4cbc68 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:12.659040 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-d89166e9-c163-45ef-a176-fca6bfda9953 admin admin] The token cache is being invalidate because project 3a5c79ca08614d7ba9cad0da6f32cccf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:12.659561 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-d89166e9-c163-45ef-a176-fca6bfda9953 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:12.660453 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-d89166e9-c163-45ef-a176-fca6bfda9953 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:12.662578 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 723/1443] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:12 2022] DELETE /identity/v3/projects/3a5c79ca08614d7ba9cad0da6f32cccf => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:51:12.666419 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89166e9-c163-45ef-a176-fca6bfda9953 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:12.667914 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-d89166e9-c163-45ef-a176-fca6bfda9953 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:12.681375 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89166e9-c163-45ef-a176-fca6bfda9953 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:12.686489 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e069a-3122-4710-9d9e-d02053ff6094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:12.687192 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e069a-3122-4710-9d9e-d02053ff6094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:12.687560 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e069a-3122-4710-9d9e-d02053ff6094 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:12.688053 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e069a-3122-4710-9d9e-d02053ff6094 admin admin] PATH_INFO: `/v3/users/3f9e4c1a43084fe58666277e03958238` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:12.689253 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e069a-3122-4710-9d9e-d02053ff6094 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f9e4c1a43084fe58666277e03958238)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:12.691205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e069a-3122-4710-9d9e-d02053ff6094 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:12.761816 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 721/1444] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:12 2022] POST /identity/v3/users => generated 295 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:12.762971 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f5ce8-d9a9-45a5-b4ed-3824bb4cbc68 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:12.764747 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-dd0f5ce8-d9a9-45a5-b4ed-3824bb4cbc68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:12.777367 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f5ce8-d9a9-45a5-b4ed-3824bb4cbc68 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:12.783731 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def4b31-3e5f-46a3-bb85-f95bf8bdbbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:12.785096 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def4b31-3e5f-46a3-bb85-f95bf8bdbbdd admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:12.785560 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def4b31-3e5f-46a3-bb85-f95bf8bdbbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:12.785953 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def4b31-3e5f-46a3-bb85-f95bf8bdbbdd admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:12.786924 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def4b31-3e5f-46a3-bb85-f95bf8bdbbdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:12.788823 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0def4b31-3e5f-46a3-bb85-f95bf8bdbbdd admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:12.803144 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 722/1445] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:12.804563 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def4b31-3e5f-46a3-bb85-f95bf8bdbbdd admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:12.815681 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def4b31-3e5f-46a3-bb85-f95bf8bdbbdd admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:12.820938 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894c45c-aedf-423f-9ca0-4a3d36c2b91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:12.821539 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894c45c-aedf-423f-9ca0-4a3d36c2b91f admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:12.821816 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894c45c-aedf-423f-9ca0-4a3d36c2b91f admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:12.822086 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894c45c-aedf-423f-9ca0-4a3d36c2b91f admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:12.822899 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7894c45c-aedf-423f-9ca0-4a3d36c2b91f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:12.824998 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7894c45c-aedf-423f-9ca0-4a3d36c2b91f admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:12.839888 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 723/1446] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:51:12.840912 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894c45c-aedf-423f-9ca0-4a3d36c2b91f admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:12.851102 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7894c45c-aedf-423f-9ca0-4a3d36c2b91f admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:12.856066 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429e8a5-c669-464a-9aad-2a9a38031179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:12.856818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429e8a5-c669-464a-9aad-2a9a38031179 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:12.857099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429e8a5-c669-464a-9aad-2a9a38031179 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:12.857369 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429e8a5-c669-464a-9aad-2a9a38031179 admin admin] PATH_INFO: `/v3/projects/01a577eb7cd84f5c8beaf0ffc7f6310d/users/2be12501fe304bb7a2ab6bb58b047b9c/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:12.868041 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:12.877501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c429e8a5-c669-464a-9aad-2a9a38031179 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01a577eb7cd84f5c8beaf0ffc7f6310d, user_id=2be12501fe304bb7a2ab6bb58b047b9c, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:12.880900 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c429e8a5-c669-464a-9aad-2a9a38031179 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:12.898488 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-5e4e069a-3122-4710-9d9e-d02053ff6094 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f9e4c1a43084fe58666277e03958238'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:12.981384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 724/1447] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:12 2022] PUT /identity/v3/projects/01a577eb7cd84f5c8beaf0ffc7f6310d/users/2be12501fe304bb7a2ab6bb58b047b9c/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:12.986746 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-5e4e069a-3122-4710-9d9e-d02053ff6094 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f9e4c1a43084fe58666277e03958238'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:12.991974 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e10e1b-2b02-44d8-8b94-983fc08d12a9 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:12.992418 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e10e1b-2b02-44d8-8b94-983fc08d12a9 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:12.992700 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e10e1b-2b02-44d8-8b94-983fc08d12a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:13.022501 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-5e4e069a-3122-4710-9d9e-d02053ff6094 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f9e4c1a43084fe58666277e03958238'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:13.023262 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-5e4e069a-3122-4710-9d9e-d02053ff6094 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f9e4c1a43084fe58666277e03958238'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:13.024049 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-5e4e069a-3122-4710-9d9e-d02053ff6094 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f9e4c1a43084fe58666277e03958238'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:13.029134 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-f8e10e1b-2b02-44d8-8b94-983fc08d12a9 None None] MFA Rules not processed for user `2be12501fe304bb7a2ab6bb58b047b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:13.030402 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-f8e10e1b-2b02-44d8-8b94-983fc08d12a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:13.069009 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 724/1448] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:51:12 2022] DELETE /identity/v3/users/3f9e4c1a43084fe58666277e03958238 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:13.142367 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 725/1449] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:12 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:51:13.147430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e069a-3122-4710-9d9e-d02053ff6094 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:13.149726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-5e4e069a-3122-4710-9d9e-d02053ff6094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:13.247413 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e069a-3122-4710-9d9e-d02053ff6094 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:13.252305 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2acdee-5cb9-4a1d-a217-920519a86199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:13.252934 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2acdee-5cb9-4a1d-a217-920519a86199 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:13.253217 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2acdee-5cb9-4a1d-a217-920519a86199 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:13.253540 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2acdee-5cb9-4a1d-a217-920519a86199 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:13.254175 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2acdee-5cb9-4a1d-a217-920519a86199 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:13.256906 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2acdee-5cb9-4a1d-a217-920519a86199 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:13.381878 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 725/1450] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:13 2022] POST /identity/v3/projects => generated 389 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:13.389270 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e10e1b-2b02-44d8-8b94-983fc08d12a9 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:13.526821 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e10e1b-2b02-44d8-8b94-983fc08d12a9 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:13.710726 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920a79a-b9cf-49ef-8e35-31cd493ba307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:13.711453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920a79a-b9cf-49ef-8e35-31cd493ba307 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:13.711782 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920a79a-b9cf-49ef-8e35-31cd493ba307 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:13.712095 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6920a79a-b9cf-49ef-8e35-31cd493ba307 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:13.712754 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6920a79a-b9cf-49ef-8e35-31cd493ba307 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:13.715235 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6920a79a-b9cf-49ef-8e35-31cd493ba307 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:13.845922 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 726/1451] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:13 2022] POST /identity/v3/users => generated 301 bytes in 460 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:51:13.846633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2acdee-5cb9-4a1d-a217-920519a86199 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:13.873492 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2acdee-5cb9-4a1d-a217-920519a86199 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:13.919101 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4138df-91c9-4b5e-8cd0-6618177bc9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:13.919872 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4138df-91c9-4b5e-8cd0-6618177bc9d8 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:13.920188 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4138df-91c9-4b5e-8cd0-6618177bc9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:13.920754 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4138df-91c9-4b5e-8cd0-6618177bc9d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:13.925208 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4138df-91c9-4b5e-8cd0-6618177bc9d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:13.928928 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4138df-91c9-4b5e-8cd0-6618177bc9d8 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:13.970137 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920a79a-b9cf-49ef-8e35-31cd493ba307 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:13.994204 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 726/1452] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:51:13.998440 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6920a79a-b9cf-49ef-8e35-31cd493ba307 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:14.021242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826d9f3-bc70-4b2c-82b9-fe191435fea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:14.021949 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826d9f3-bc70-4b2c-82b9-fe191435fea6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:14.022235 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826d9f3-bc70-4b2c-82b9-fe191435fea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:14.022515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826d9f3-bc70-4b2c-82b9-fe191435fea6 admin admin] PATH_INFO: `/v3/projects/05aebbe7ff8545cfb177be74da37b362/users/01e26617703749e99bafe4fdb39bd007/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:14.062667 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:14.075773 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8826d9f3-bc70-4b2c-82b9-fe191435fea6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05aebbe7ff8545cfb177be74da37b362, user_id=01e26617703749e99bafe4fdb39bd007, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:14.079548 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8826d9f3-bc70-4b2c-82b9-fe191435fea6 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:14.229627 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 727/1453] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:13 2022] PUT /identity/v3/projects/05aebbe7ff8545cfb177be74da37b362/users/01e26617703749e99bafe4fdb39bd007/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:51:14.233850 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4138df-91c9-4b5e-8cd0-6618177bc9d8 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:14.251550 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4138df-91c9-4b5e-8cd0-6618177bc9d8 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:14.390898 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:14.458824 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b756b-4462-48a0-8311-b61aecdee486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:14.459633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b756b-4462-48a0-8311-b61aecdee486 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:14.459961 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b756b-4462-48a0-8311-b61aecdee486 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:14.460209 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b756b-4462-48a0-8311-b61aecdee486 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:14.460963 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b756b-4462-48a0-8311-b61aecdee486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:14.463202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0b756b-4462-48a0-8311-b61aecdee486 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:14.520028 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 727/1454] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:14 2022] GET /identity/v3/roles => generated 1420 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:51:14.525594 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826d9f3-bc70-4b2c-82b9-fe191435fea6 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:14.563321 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826d9f3-bc70-4b2c-82b9-fe191435fea6 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:14.598759 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aee7e2-6db7-4bc5-acce-968bb39ecf02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:14.602264 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aee7e2-6db7-4bc5-acce-968bb39ecf02 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:14.605064 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aee7e2-6db7-4bc5-acce-968bb39ecf02 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:14.610879 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aee7e2-6db7-4bc5-acce-968bb39ecf02 admin admin] PATH_INFO: `/v3/projects/05aebbe7ff8545cfb177be74da37b362/users/01e26617703749e99bafe4fdb39bd007/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:14.620445 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aee7e2-6db7-4bc5-acce-968bb39ecf02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05aebbe7ff8545cfb177be74da37b362, user_id=01e26617703749e99bafe4fdb39bd007, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:14.623137 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aee7e2-6db7-4bc5-acce-968bb39ecf02 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:14.711312 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b756b-4462-48a0-8311-b61aecdee486 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:14.714270 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-4a0b756b-4462-48a0-8311-b61aecdee486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:14.751468 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 728/1455] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:14 2022] PUT /identity/v3/projects/05aebbe7ff8545cfb177be74da37b362/users/01e26617703749e99bafe4fdb39bd007/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:14.759545 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56058300-d950-47ef-b523-e37a77f1de97 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:14.759867 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56058300-d950-47ef-b523-e37a77f1de97 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:14.760144 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56058300-d950-47ef-b523-e37a77f1de97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:14.799295 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b756b-4462-48a0-8311-b61aecdee486 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:14.880484 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-56058300-d950-47ef-b523-e37a77f1de97 None None] MFA Rules not processed for user `01e26617703749e99bafe4fdb39bd007`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:14.892580 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-56058300-d950-47ef-b523-e37a77f1de97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:15.008477 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e7868c-2329-4f90-a99b-74744699cb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:15.013533 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7868c-2329-4f90-a99b-74744699cb29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:15.014256 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7868c-2329-4f90-a99b-74744699cb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:15.015868 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e7868c-2329-4f90-a99b-74744699cb29 admin admin] PATH_INFO: `/v3/projects/ef1cc4bcedbe45a3af49017162634616` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:15.018384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7868c-2329-4f90-a99b-74744699cb29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef1cc4bcedbe45a3af49017162634616)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:15.024142 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e7868c-2329-4f90-a99b-74744699cb29 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:15.065045 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 729/1456] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:14 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:15.237320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-25e7868c-2329-4f90-a99b-74744699cb29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef1cc4bcedbe45a3af49017162634616', 'request_id': 'req-25e7868c-2329-4f90-a99b-74744699cb29'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:15.260147 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56058300-d950-47ef-b523-e37a77f1de97 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:15.261788 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-56058300-d950-47ef-b523-e37a77f1de97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:15.337059 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-25e7868c-2329-4f90-a99b-74744699cb29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef1cc4bcedbe45a3af49017162634616', 'request_id': 'req-25e7868c-2329-4f90-a99b-74744699cb29'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:15.346565 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-25e7868c-2329-4f90-a99b-74744699cb29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef1cc4bcedbe45a3af49017162634616', 'request_id': 'req-25e7868c-2329-4f90-a99b-74744699cb29'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:15.383306 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-25e7868c-2329-4f90-a99b-74744699cb29 admin admin] The token cache is being invalidate because project ef1cc4bcedbe45a3af49017162634616 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:15.383611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-25e7868c-2329-4f90-a99b-74744699cb29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:15.386126 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-25e7868c-2329-4f90-a99b-74744699cb29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:15.396416 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 728/1457] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:14 2022] DELETE /identity/v3/projects/ef1cc4bcedbe45a3af49017162634616 => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:15.397911 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10698ba2-173c-4728-8546-b851b6179584 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:15.399169 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10698ba2-173c-4728-8546-b851b6179584 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:15.399456 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10698ba2-173c-4728-8546-b851b6179584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:15.460691 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56058300-d950-47ef-b523-e37a77f1de97 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:15.460789 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-10698ba2-173c-4728-8546-b851b6179584 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:15.460789 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-10698ba2-173c-4728-8546-b851b6179584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:15.558862 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a65ddf-f996-4ea1-bf72-a504bb41e44a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:15.560326 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a65ddf-f996-4ea1-bf72-a504bb41e44a service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:15.560628 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a65ddf-f996-4ea1-bf72-a504bb41e44a service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:15.560910 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a65ddf-f996-4ea1-bf72-a504bb41e44a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:15.564157 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-b2a65ddf-f996-4ea1-bf72-a504bb41e44a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:15.588964 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:15.609002 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 729/1458] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:15.616982 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e6a52-5e43-42f7-b799-9cbf763312ff None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:15.617255 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e6a52-5e43-42f7-b799-9cbf763312ff None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:15.617530 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e6a52-5e43-42f7-b799-9cbf763312ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:15.628411 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:15.679225 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a65ddf-f996-4ea1-bf72-a504bb41e44a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:15.681268 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a65ddf-f996-4ea1-bf72-a504bb41e44a service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:15.711437 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 730/1459] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:15 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:51:16.058749 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-845e6a52-5e43-42f7-b799-9cbf763312ff None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:16.060335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-845e6a52-5e43-42f7-b799-9cbf763312ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:16.096039 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 730/1460] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:16.101954 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a65ddf-f996-4ea1-bf72-a504bb41e44a service cinder] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:16.108564 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a65ddf-f996-4ea1-bf72-a504bb41e44a service cinder] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:16.115204 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3373d-b4fd-417a-9fbf-16306653e351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:16.115808 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3373d-b4fd-417a-9fbf-16306653e351 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:16.116097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3373d-b4fd-417a-9fbf-16306653e351 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:16.116392 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3373d-b4fd-417a-9fbf-16306653e351 admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:16.117038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3373d-b4fd-417a-9fbf-16306653e351 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:51:16.117342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3373d-b4fd-417a-9fbf-16306653e351 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:16.119226 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3373d-b4fd-417a-9fbf-16306653e351 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:16.133591 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 731/1461] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:51:16.146462 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3373d-b4fd-417a-9fbf-16306653e351 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:16.148336 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-a4d3373d-b4fd-417a-9fbf-16306653e351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:16.156911 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e6a52-5e43-42f7-b799-9cbf763312ff None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:16.169135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e6a52-5e43-42f7-b799-9cbf763312ff None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:16.175475 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c454b-c0ce-4c9e-bdc8-04c3dc36d9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:16.177096 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c454b-c0ce-4c9e-bdc8-04c3dc36d9f9 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:16.177096 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c454b-c0ce-4c9e-bdc8-04c3dc36d9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:16.177286 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c454b-c0ce-4c9e-bdc8-04c3dc36d9f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:16.177787 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c454b-c0ce-4c9e-bdc8-04c3dc36d9f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:16.178696 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3373d-b4fd-417a-9fbf-16306653e351 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:16.180490 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4c454b-c0ce-4c9e-bdc8-04c3dc36d9f9 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:16.184136 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093acbe-0a5f-4082-abfa-46c414137848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:16.184867 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093acbe-0a5f-4082-abfa-46c414137848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:16.185443 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093acbe-0a5f-4082-abfa-46c414137848 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:16.185796 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093acbe-0a5f-4082-abfa-46c414137848 admin admin] PATH_INFO: `/v3/users/2be12501fe304bb7a2ab6bb58b047b9c` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:16.187684 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093acbe-0a5f-4082-abfa-46c414137848 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2be12501fe304bb7a2ab6bb58b047b9c)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:16.189864 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093acbe-0a5f-4082-abfa-46c414137848 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:16.276693 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 731/1462] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:16 2022] POST /identity/v3/projects => generated 369 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:16.278782 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c454b-c0ce-4c9e-bdc8-04c3dc36d9f9 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:16.369000 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4c454b-c0ce-4c9e-bdc8-04c3dc36d9f9 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:16.374156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a1165-4c64-47e4-a872-da7d9e2cb297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:16.375532 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a1165-4c64-47e4-a872-da7d9e2cb297 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:16.375532 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a1165-4c64-47e4-a872-da7d9e2cb297 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:16.375532 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a1165-4c64-47e4-a872-da7d9e2cb297 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:16.376120 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5a1165-4c64-47e4-a872-da7d9e2cb297 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:16.378028 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5a1165-4c64-47e4-a872-da7d9e2cb297 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:16.404649 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-4093acbe-0a5f-4082-abfa-46c414137848 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2be12501fe304bb7a2ab6bb58b047b9c'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:16.405851 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-4093acbe-0a5f-4082-abfa-46c414137848 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2be12501fe304bb7a2ab6bb58b047b9c'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:16.496516 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 732/1463] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:16 2022] POST /identity/v3/users => generated 285 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:16.500921 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a1165-4c64-47e4-a872-da7d9e2cb297 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:16.503057 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-8c5a1165-4c64-47e4-a872-da7d9e2cb297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:16.506782 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-4093acbe-0a5f-4082-abfa-46c414137848 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2be12501fe304bb7a2ab6bb58b047b9c'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:16.531674 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-4093acbe-0a5f-4082-abfa-46c414137848 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2be12501fe304bb7a2ab6bb58b047b9c'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:16.533100 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-4093acbe-0a5f-4082-abfa-46c414137848 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2be12501fe304bb7a2ab6bb58b047b9c'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:16.540772 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a1165-4c64-47e4-a872-da7d9e2cb297 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:16.559223 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955608d5-4137-4530-8f02-e3451e869e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:16.559886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955608d5-4137-4530-8f02-e3451e869e19 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:16.560167 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955608d5-4137-4530-8f02-e3451e869e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:16.560435 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955608d5-4137-4530-8f02-e3451e869e19 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:16.561085 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955608d5-4137-4530-8f02-e3451e869e19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:16.562755 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955608d5-4137-4530-8f02-e3451e869e19 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:16.574848 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 732/1464] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:51:16 2022] DELETE /identity/v3/users/2be12501fe304bb7a2ab6bb58b047b9c => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:16.597568 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22252711-acdc-46d7-9ede-64fcc172493c None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:16.606235 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22252711-acdc-46d7-9ede-64fcc172493c None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:16.606464 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 733/1465] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:51:16.606464 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955608d5-4137-4530-8f02-e3451e869e19 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:16.606686 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22252711-acdc-46d7-9ede-64fcc172493c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:16.632938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955608d5-4137-4530-8f02-e3451e869e19 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:16.659730 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-22252711-acdc-46d7-9ede-64fcc172493c None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:16.662600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-22252711-acdc-46d7-9ede-64fcc172493c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:16.749343 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa167e0-ca4c-4797-93ca-ab0c18f91593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:16.750228 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa167e0-ca4c-4797-93ca-ab0c18f91593 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:16.750715 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa167e0-ca4c-4797-93ca-ab0c18f91593 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:16.750991 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa167e0-ca4c-4797-93ca-ab0c18f91593 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:16.751917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa167e0-ca4c-4797-93ca-ab0c18f91593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:16.755634 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa167e0-ca4c-4797-93ca-ab0c18f91593 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:16.781998 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 733/1466] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:16.792824 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22252711-acdc-46d7-9ede-64fcc172493c None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:16.796863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 734/1467] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:16 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:16.800630 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-22252711-acdc-46d7-9ede-64fcc172493c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:16.800766 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa167e0-ca4c-4797-93ca-ab0c18f91593 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:16.812114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa167e0-ca4c-4797-93ca-ab0c18f91593 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:16.817999 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830dbe6-9a39-4530-a838-3b9f137b1f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:16.818672 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830dbe6-9a39-4530-a838-3b9f137b1f4a admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:16.818950 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830dbe6-9a39-4530-a838-3b9f137b1f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:16.819284 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830dbe6-9a39-4530-a838-3b9f137b1f4a admin admin] PATH_INFO: `/v3/projects/6d557d2ff4424da2916a9aeddc602f48/users/7440bef61da145d982381f9dfc320ff6/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:16.839331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:16.841225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830dbe6-9a39-4530-a838-3b9f137b1f4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d557d2ff4424da2916a9aeddc602f48, user_id=7440bef61da145d982381f9dfc320ff6, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:16.844373 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830dbe6-9a39-4530-a838-3b9f137b1f4a admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:16.888336 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22252711-acdc-46d7-9ede-64fcc172493c None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:16.895775 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd1980-cb9f-4676-86a6-1ceafb6540d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:16.895775 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd1980-cb9f-4676-86a6-1ceafb6540d0 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:16.895775 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd1980-cb9f-4676-86a6-1ceafb6540d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:16.896036 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd1980-cb9f-4676-86a6-1ceafb6540d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:16.908777 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd1980-cb9f-4676-86a6-1ceafb6540d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:16.911400 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd1980-cb9f-4676-86a6-1ceafb6540d0 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:16.928206 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 735/1468] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:16 2022] PUT /identity/v3/projects/6d557d2ff4424da2916a9aeddc602f48/users/7440bef61da145d982381f9dfc320ff6/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:51:16.929920 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5173e1e-7004-4b72-9ced-dc90bff2cbe7 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:16.930214 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5173e1e-7004-4b72-9ced-dc90bff2cbe7 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:16.932639 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5173e1e-7004-4b72-9ced-dc90bff2cbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:16.944264 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 734/1469] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:16.976986 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-b5173e1e-7004-4b72-9ced-dc90bff2cbe7 None None] MFA Rules not processed for user `7440bef61da145d982381f9dfc320ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:16.986019 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-b5173e1e-7004-4b72-9ced-dc90bff2cbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:16.995476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:17.054837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 736/1470] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:17.059821 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd1980-cb9f-4676-86a6-1ceafb6540d0 service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:17.073011 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd1980-cb9f-4676-86a6-1ceafb6540d0 service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:17.150232 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a219015-2144-44af-8b40-4db5458eb895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:17.150999 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a219015-2144-44af-8b40-4db5458eb895 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:17.151255 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a219015-2144-44af-8b40-4db5458eb895 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:17.151527 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a219015-2144-44af-8b40-4db5458eb895 admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:17.152288 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a219015-2144-44af-8b40-4db5458eb895 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:17.156378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a219015-2144-44af-8b40-4db5458eb895 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:17.287085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 735/1471] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:17 2022] POST /identity/v3/projects => generated 369 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:17.290862 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5173e1e-7004-4b72-9ced-dc90bff2cbe7 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:17.308766 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5173e1e-7004-4b72-9ced-dc90bff2cbe7 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:17.403848 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e0ccb-6f6d-426e-a8e6-2c1f383a4243 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:17.404611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e0ccb-6f6d-426e-a8e6-2c1f383a4243 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:17.405076 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3e0ccb-6f6d-426e-a8e6-2c1f383a4243 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:17.406698 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 736/1472] 10.222.0.17 () {54 vars in 921 bytes} [Mon Oct 17 03:51:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:51:17.411062 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67dafbb-dec2-4b27-9cb8-164a2eaf5787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:17.411958 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dafbb-dec2-4b27-9cb8-164a2eaf5787 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:17.412307 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dafbb-dec2-4b27-9cb8-164a2eaf5787 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:17.412967 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67dafbb-dec2-4b27-9cb8-164a2eaf5787 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:17.413630 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67dafbb-dec2-4b27-9cb8-164a2eaf5787 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:17.416552 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67dafbb-dec2-4b27-9cb8-164a2eaf5787 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:17.435788 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319eac3e-ac20-4d60-a82b-de4db3b5b424 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:17.436568 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319eac3e-ac20-4d60-a82b-de4db3b5b424 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:17.436568 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319eac3e-ac20-4d60-a82b-de4db3b5b424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:17.464649 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-319eac3e-ac20-4d60-a82b-de4db3b5b424 None None] MFA Rules not processed for user `d961b751763a4e49b5d4ca4c795c5bf6`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:17.466327 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-319eac3e-ac20-4d60-a82b-de4db3b5b424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:17.524406 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 737/1473] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:17 2022] POST /identity/v3/users => generated 291 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:17.528076 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67dafbb-dec2-4b27-9cb8-164a2eaf5787 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:17.537741 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67dafbb-dec2-4b27-9cb8-164a2eaf5787 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:17.541103 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 737/1474] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Oct 17 03:51:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:17.543810 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95415031-3ce2-4677-b5dd-9957bfbf9f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:17.546717 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95415031-3ce2-4677-b5dd-9957bfbf9f25 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:17.546717 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95415031-3ce2-4677-b5dd-9957bfbf9f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:17.546717 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95415031-3ce2-4677-b5dd-9957bfbf9f25 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:17.546717 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95415031-3ce2-4677-b5dd-9957bfbf9f25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:17.547203 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95415031-3ce2-4677-b5dd-9957bfbf9f25 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:17.554968 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319eac3e-ac20-4d60-a82b-de4db3b5b424 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:17.556661 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-319eac3e-ac20-4d60-a82b-de4db3b5b424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:17.558686 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 738/1475] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:51:17.562564 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95415031-3ce2-4677-b5dd-9957bfbf9f25 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:17.571324 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319eac3e-ac20-4d60-a82b-de4db3b5b424 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:17.572882 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95415031-3ce2-4677-b5dd-9957bfbf9f25 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:17.576957 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac315b4f-cca9-4695-b791-22968ddbc757 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:17.577475 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e726dab-a38e-4b90-a1dc-f5ccdff4aca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:17.577598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac315b4f-cca9-4695-b791-22968ddbc757 service nova] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:17.577825 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac315b4f-cca9-4695-b791-22968ddbc757 service nova] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:17.578098 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac315b4f-cca9-4695-b791-22968ddbc757 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:17.578187 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e726dab-a38e-4b90-a1dc-f5ccdff4aca5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:17.578451 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e726dab-a38e-4b90-a1dc-f5ccdff4aca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:17.578765 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e726dab-a38e-4b90-a1dc-f5ccdff4aca5 admin admin] PATH_INFO: `/v3/projects/e0374528600d4afcb70c006e9fb44bce/users/e9979a5a177e43b88cdf52f7e15ffb49/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:17.589745 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac315b4f-cca9-4695-b791-22968ddbc757 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:17.591385 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac315b4f-cca9-4695-b791-22968ddbc757 service nova] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:17.594828 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e726dab-a38e-4b90-a1dc-f5ccdff4aca5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0374528600d4afcb70c006e9fb44bce, user_id=e9979a5a177e43b88cdf52f7e15ffb49, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:17.598388 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e726dab-a38e-4b90-a1dc-f5ccdff4aca5 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:17.602578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 738/1476] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:51:17.608973 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac315b4f-cca9-4695-b791-22968ddbc757 service nova] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:17.610228 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-ac315b4f-cca9-4695-b791-22968ddbc757 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:17.625079 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac315b4f-cca9-4695-b791-22968ddbc757 service nova] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:17.633876 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f14c1-e3c1-4e01-80bf-caccf3c47ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:17.634547 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f14c1-e3c1-4e01-80bf-caccf3c47ba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:17.634825 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f14c1-e3c1-4e01-80bf-caccf3c47ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:17.635095 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f14c1-e3c1-4e01-80bf-caccf3c47ba6 admin admin] PATH_INFO: `/v3/projects/01a577eb7cd84f5c8beaf0ffc7f6310d` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:17.636329 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9f14c1-e3c1-4e01-80bf-caccf3c47ba6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01a577eb7cd84f5c8beaf0ffc7f6310d)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:17.638279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9f14c1-e3c1-4e01-80bf-caccf3c47ba6 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:17.687892 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 739/1477] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:17 2022] PUT /identity/v3/projects/e0374528600d4afcb70c006e9fb44bce/users/e9979a5a177e43b88cdf52f7e15ffb49/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:51:17.688560 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e726dab-a38e-4b90-a1dc-f5ccdff4aca5 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:17.749551 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e726dab-a38e-4b90-a1dc-f5ccdff4aca5 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:17.759392 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-df9f14c1-e3c1-4e01-80bf-caccf3c47ba6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01a577eb7cd84f5c8beaf0ffc7f6310d', 'request_id': 'req-df9f14c1-e3c1-4e01-80bf-caccf3c47ba6'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:17.804347 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e53528-e806-4508-9777-fec2e8b330fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:17.804951 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e53528-e806-4508-9777-fec2e8b330fe admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:17.805230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e53528-e806-4508-9777-fec2e8b330fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:17.805501 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e53528-e806-4508-9777-fec2e8b330fe admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:17.806243 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e53528-e806-4508-9777-fec2e8b330fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:17.807705 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e53528-e806-4508-9777-fec2e8b330fe admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:17.817792 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 740/1478] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:51:17.821929 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e53528-e806-4508-9777-fec2e8b330fe admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:17.831633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e53528-e806-4508-9777-fec2e8b330fe admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:17.836773 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a55f9-9a0f-4d9d-aef5-afca9ca4e28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:17.837396 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a55f9-9a0f-4d9d-aef5-afca9ca4e28b admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:17.837674 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a55f9-9a0f-4d9d-aef5-afca9ca4e28b admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:17.837762 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-df9f14c1-e3c1-4e01-80bf-caccf3c47ba6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01a577eb7cd84f5c8beaf0ffc7f6310d', 'request_id': 'req-df9f14c1-e3c1-4e01-80bf-caccf3c47ba6'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:17.837948 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a55f9-9a0f-4d9d-aef5-afca9ca4e28b admin admin] PATH_INFO: `/v3/projects/e0374528600d4afcb70c006e9fb44bce/users/e9979a5a177e43b88cdf52f7e15ffb49/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:17.839909 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a55f9-9a0f-4d9d-aef5-afca9ca4e28b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0374528600d4afcb70c006e9fb44bce, user_id=e9979a5a177e43b88cdf52f7e15ffb49, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:17.842532 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a55f9-9a0f-4d9d-aef5-afca9ca4e28b admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:17.850559 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-df9f14c1-e3c1-4e01-80bf-caccf3c47ba6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01a577eb7cd84f5c8beaf0ffc7f6310d', 'request_id': 'req-df9f14c1-e3c1-4e01-80bf-caccf3c47ba6'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:17.853202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 741/1479] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:17 2022] PUT /identity/v3/projects/e0374528600d4afcb70c006e9fb44bce/users/e9979a5a177e43b88cdf52f7e15ffb49/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:51:17.857235 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-df9f14c1-e3c1-4e01-80bf-caccf3c47ba6 admin admin] The token cache is being invalidate because project 01a577eb7cd84f5c8beaf0ffc7f6310d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:17.857549 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-df9f14c1-e3c1-4e01-80bf-caccf3c47ba6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:17.858250 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-df9f14c1-e3c1-4e01-80bf-caccf3c47ba6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:17.859922 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 739/1480] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:17 2022] DELETE /identity/v3/projects/01a577eb7cd84f5c8beaf0ffc7f6310d => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:17.863500 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f14c1-e3c1-4e01-80bf-caccf3c47ba6 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:17.864148 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964b156-f043-4f6b-a310-7d0ebd6e651b None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:17.864435 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964b156-f043-4f6b-a310-7d0ebd6e651b None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:17.864707 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964b156-f043-4f6b-a310-7d0ebd6e651b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:17.865010 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-df9f14c1-e3c1-4e01-80bf-caccf3c47ba6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:17.878426 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f14c1-e3c1-4e01-80bf-caccf3c47ba6 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:17.896010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-6964b156-f043-4f6b-a310-7d0ebd6e651b None None] MFA Rules not processed for user `e9979a5a177e43b88cdf52f7e15ffb49`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:17.897324 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-6964b156-f043-4f6b-a310-7d0ebd6e651b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:17.937585 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7377f14-af4a-4792-80ab-39f8ab058eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:17.938223 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7377f14-af4a-4792-80ab-39f8ab058eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:17.938504 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7377f14-af4a-4792-80ab-39f8ab058eba admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:17.938778 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7377f14-af4a-4792-80ab-39f8ab058eba admin admin] PATH_INFO: `/v3/users/01e26617703749e99bafe4fdb39bd007` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:17.940000 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7377f14-af4a-4792-80ab-39f8ab058eba admin admin] RBAC: Authorizing `identity:delete_user(user_id=01e26617703749e99bafe4fdb39bd007)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:17.942418 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7377f14-af4a-4792-80ab-39f8ab058eba admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:17.965792 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 742/1481] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:17 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:51:17.977593 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964b156-f043-4f6b-a310-7d0ebd6e651b None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:17.978918 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-6964b156-f043-4f6b-a310-7d0ebd6e651b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:18.033585 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964b156-f043-4f6b-a310-7d0ebd6e651b None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:18.037248 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ed65b-d926-4a9d-a49f-fe2696459eb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:18.037845 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ed65b-d926-4a9d-a49f-fe2696459eb1 service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:18.038149 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ed65b-d926-4a9d-a49f-fe2696459eb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:18.038417 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2ed65b-d926-4a9d-a49f-fe2696459eb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:18.047088 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2ed65b-d926-4a9d-a49f-fe2696459eb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:18.048519 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2ed65b-d926-4a9d-a49f-fe2696459eb1 service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:18.057854 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 743/1482] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:17 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:51:18.126229 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-c7377f14-af4a-4792-80ab-39f8ab058eba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01e26617703749e99bafe4fdb39bd007'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:18.126935 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-c7377f14-af4a-4792-80ab-39f8ab058eba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01e26617703749e99bafe4fdb39bd007'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:18.210011 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-c7377f14-af4a-4792-80ab-39f8ab058eba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01e26617703749e99bafe4fdb39bd007'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:18.231525 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-c7377f14-af4a-4792-80ab-39f8ab058eba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01e26617703749e99bafe4fdb39bd007'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:18.231972 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ed65b-d926-4a9d-a49f-fe2696459eb1 service cinder] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:18.232692 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-c7377f14-af4a-4792-80ab-39f8ab058eba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01e26617703749e99bafe4fdb39bd007'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:18.236329 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-ee2ed65b-d926-4a9d-a49f-fe2696459eb1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:18.245754 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2ed65b-d926-4a9d-a49f-fe2696459eb1 service cinder] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:18.252121 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfe7eec-5be1-47b6-98be-d1cbadb1e876 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:18.253158 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe7eec-5be1-47b6-98be-d1cbadb1e876 service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:18.253617 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe7eec-5be1-47b6-98be-d1cbadb1e876 service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:18.254724 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfe7eec-5be1-47b6-98be-d1cbadb1e876 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:18.259292 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-bbfe7eec-5be1-47b6-98be-d1cbadb1e876 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:18.261832 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 740/1483] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:51:17 2022] DELETE /identity/v3/users/01e26617703749e99bafe4fdb39bd007 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:18.326657 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfe7eec-5be1-47b6-98be-d1cbadb1e876 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:18.328730 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfe7eec-5be1-47b6-98be-d1cbadb1e876 service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:18.340913 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 744/1484] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:51:18.419345 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:19.060112 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:19.093030 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7377f14-af4a-4792-80ab-39f8ab058eba admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:19.094931 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-c7377f14-af4a-4792-80ab-39f8ab058eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:19.186138 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7377f14-af4a-4792-80ab-39f8ab058eba admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:19.191621 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab63852f-ed19-4b8f-96fa-83634ed8b989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:19.192968 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63852f-ed19-4b8f-96fa-83634ed8b989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:19.193412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63852f-ed19-4b8f-96fa-83634ed8b989 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:19.193583 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63852f-ed19-4b8f-96fa-83634ed8b989 admin admin] PATH_INFO: `/v3/projects/05aebbe7ff8545cfb177be74da37b362` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:19.195093 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab63852f-ed19-4b8f-96fa-83634ed8b989 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05aebbe7ff8545cfb177be74da37b362)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:19.197034 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab63852f-ed19-4b8f-96fa-83634ed8b989 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:19.333179 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ab63852f-ed19-4b8f-96fa-83634ed8b989 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05aebbe7ff8545cfb177be74da37b362', 'request_id': 'req-ab63852f-ed19-4b8f-96fa-83634ed8b989'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:19.428747 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ab63852f-ed19-4b8f-96fa-83634ed8b989 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05aebbe7ff8545cfb177be74da37b362', 'request_id': 'req-ab63852f-ed19-4b8f-96fa-83634ed8b989'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:19.442351 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ab63852f-ed19-4b8f-96fa-83634ed8b989 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05aebbe7ff8545cfb177be74da37b362', 'request_id': 'req-ab63852f-ed19-4b8f-96fa-83634ed8b989'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:19.467705 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ab63852f-ed19-4b8f-96fa-83634ed8b989 admin admin] The token cache is being invalidate because project 05aebbe7ff8545cfb177be74da37b362 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:19.467705 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ab63852f-ed19-4b8f-96fa-83634ed8b989 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:19.477039 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ab63852f-ed19-4b8f-96fa-83634ed8b989 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:19.483859 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 741/1485] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:19 2022] DELETE /identity/v3/projects/05aebbe7ff8545cfb177be74da37b362 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:19.488841 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4f8c4-d302-4e30-b155-93a27db72e25 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:19.489812 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4f8c4-d302-4e30-b155-93a27db72e25 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:19.490296 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4f8c4-d302-4e30-b155-93a27db72e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:19.519268 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-a5e4f8c4-d302-4e30-b155-93a27db72e25 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:19.521092 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-a5e4f8c4-d302-4e30-b155-93a27db72e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:19.617044 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 745/1486] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:19.626005 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c167728-71e9-4b9a-9811-ff6312c43bdb None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:19.626435 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c167728-71e9-4b9a-9811-ff6312c43bdb None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:19.626806 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c167728-71e9-4b9a-9811-ff6312c43bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:19.664665 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-4c167728-71e9-4b9a-9811-ff6312c43bdb None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:19.666099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-4c167728-71e9-4b9a-9811-ff6312c43bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:19.679296 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 742/1487] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:19.690140 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4f8c4-d302-4e30-b155-93a27db72e25 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:19.711024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4f8c4-d302-4e30-b155-93a27db72e25 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:19.728843 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc456f0a-a8f3-4b4d-bc77-a330e1bbb3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:19.730707 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc456f0a-a8f3-4b4d-bc77-a330e1bbb3a8 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:19.730707 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc456f0a-a8f3-4b4d-bc77-a330e1bbb3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:19.732217 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc456f0a-a8f3-4b4d-bc77-a330e1bbb3a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:19.732974 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc456f0a-a8f3-4b4d-bc77-a330e1bbb3a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:51:19.741494 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc456f0a-a8f3-4b4d-bc77-a330e1bbb3a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:19.741494 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc456f0a-a8f3-4b4d-bc77-a330e1bbb3a8 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:19.754700 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 746/1488] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:51:19.758891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c167728-71e9-4b9a-9811-ff6312c43bdb None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:19.770496 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c167728-71e9-4b9a-9811-ff6312c43bdb None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:19.781846 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b64897-6a12-4698-8d5a-7cb8f7671778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:19.782585 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b64897-6a12-4698-8d5a-7cb8f7671778 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:19.783719 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b64897-6a12-4698-8d5a-7cb8f7671778 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:19.783719 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b64897-6a12-4698-8d5a-7cb8f7671778 admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:19.786488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b64897-6a12-4698-8d5a-7cb8f7671778 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:19.788542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b64897-6a12-4698-8d5a-7cb8f7671778 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:19.922507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 743/1489] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:19 2022] POST /identity/v3/projects => generated 407 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:19.927237 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc456f0a-a8f3-4b4d-bc77-a330e1bbb3a8 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:19.951844 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc456f0a-a8f3-4b4d-bc77-a330e1bbb3a8 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:20.095625 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b64897-6a12-4698-8d5a-7cb8f7671778 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:20.097230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dbef6c-d62d-4c92-bd72-b11e4fa8733a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:20.097658 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-63b64897-6a12-4698-8d5a-7cb8f7671778 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:20.097892 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dbef6c-d62d-4c92-bd72-b11e4fa8733a admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:20.098612 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dbef6c-d62d-4c92-bd72-b11e4fa8733a admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:20.098895 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dbef6c-d62d-4c92-bd72-b11e4fa8733a admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:20.099669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dbef6c-d62d-4c92-bd72-b11e4fa8733a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:20.101808 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dbef6c-d62d-4c92-bd72-b11e4fa8733a admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:20.127080 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b64897-6a12-4698-8d5a-7cb8f7671778 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:20.145096 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f44b71-25c0-4e43-99a2-c238855dbeda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:20.145771 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f44b71-25c0-4e43-99a2-c238855dbeda admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:20.154800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f44b71-25c0-4e43-99a2-c238855dbeda admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:20.157093 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:20.159913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f44b71-25c0-4e43-99a2-c238855dbeda admin admin] PATH_INFO: `/v3/users/201facab38314abca6062b948dfd1c02` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:20.163821 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f44b71-25c0-4e43-99a2-c238855dbeda admin admin] RBAC: Authorizing `identity:delete_user(user_id=201facab38314abca6062b948dfd1c02)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:20.167586 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f44b71-25c0-4e43-99a2-c238855dbeda admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:20.235640 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 747/1490] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:19 2022] POST /identity/v3/users => generated 304 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:20.238890 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dbef6c-d62d-4c92-bd72-b11e4fa8733a admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:20.260098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dbef6c-d62d-4c92-bd72-b11e4fa8733a admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:20.267777 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081abd5b-de0e-4cf4-a0c8-13f3f2c11e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:20.268803 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081abd5b-de0e-4cf4-a0c8-13f3f2c11e5b admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:20.269682 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081abd5b-de0e-4cf4-a0c8-13f3f2c11e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:20.270747 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081abd5b-de0e-4cf4-a0c8-13f3f2c11e5b admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:20.271918 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081abd5b-de0e-4cf4-a0c8-13f3f2c11e5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:20.274567 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081abd5b-de0e-4cf4-a0c8-13f3f2c11e5b admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:20.300378 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 748/1491] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:20.306167 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081abd5b-de0e-4cf4-a0c8-13f3f2c11e5b admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:20.323431 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081abd5b-de0e-4cf4-a0c8-13f3f2c11e5b admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:20.329857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab49db-d423-4248-8699-b1ce5ac4402a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:20.330636 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab49db-d423-4248-8699-b1ce5ac4402a admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:20.331034 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab49db-d423-4248-8699-b1ce5ac4402a admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:20.331569 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdab49db-d423-4248-8699-b1ce5ac4402a admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:20.332895 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab49db-d423-4248-8699-b1ce5ac4402a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:20.336248 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdab49db-d423-4248-8699-b1ce5ac4402a admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:20.351209 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 749/1492] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:20.352430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab49db-d423-4248-8699-b1ce5ac4402a admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:20.374472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdab49db-d423-4248-8699-b1ce5ac4402a admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:20.386111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d146c043-5340-4b8b-aa70-6e7683bcf547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:20.386923 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146c043-5340-4b8b-aa70-6e7683bcf547 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:20.387339 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146c043-5340-4b8b-aa70-6e7683bcf547 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:20.387624 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d146c043-5340-4b8b-aa70-6e7683bcf547 admin admin] PATH_INFO: `/v3/projects/785f10a2c39946ae8d812e4857a1e8d1/users/f877bfbb85b24b5a8bc98516066f4164/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:20.408876 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d146c043-5340-4b8b-aa70-6e7683bcf547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=785f10a2c39946ae8d812e4857a1e8d1, user_id=f877bfbb85b24b5a8bc98516066f4164, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:20.414464 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d146c043-5340-4b8b-aa70-6e7683bcf547 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:20.498003 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 750/1493] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:20 2022] PUT /identity/v3/projects/785f10a2c39946ae8d812e4857a1e8d1/users/f877bfbb85b24b5a8bc98516066f4164/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:20.512836 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0572ce2-72ba-431d-8215-94df91f8c61f None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:20.513197 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0572ce2-72ba-431d-8215-94df91f8c61f None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:20.517704 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0572ce2-72ba-431d-8215-94df91f8c61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:20.524015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-15f44b71-25c0-4e43-99a2-c238855dbeda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '201facab38314abca6062b948dfd1c02'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:20.526847 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-15f44b71-25c0-4e43-99a2-c238855dbeda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '201facab38314abca6062b948dfd1c02'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:20.581742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-a0572ce2-72ba-431d-8215-94df91f8c61f None None] MFA Rules not processed for user `f877bfbb85b24b5a8bc98516066f4164`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:20.585200 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-a0572ce2-72ba-431d-8215-94df91f8c61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:20.632079 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-15f44b71-25c0-4e43-99a2-c238855dbeda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '201facab38314abca6062b948dfd1c02'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:20.662694 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-15f44b71-25c0-4e43-99a2-c238855dbeda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '201facab38314abca6062b948dfd1c02'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:20.663633 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-15f44b71-25c0-4e43-99a2-c238855dbeda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '201facab38314abca6062b948dfd1c02'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:20.680808 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 744/1494] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:51:20 2022] DELETE /identity/v3/users/201facab38314abca6062b948dfd1c02 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:51:20.688409 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a2430-b03e-40d2-87b4-6678a45ef6b0 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:20.689082 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a2430-b03e-40d2-87b4-6678a45ef6b0 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:20.691518 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a2430-b03e-40d2-87b4-6678a45ef6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:20.753412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-036a2430-b03e-40d2-87b4-6678a45ef6b0 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:20.757190 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-036a2430-b03e-40d2-87b4-6678a45ef6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:20.778830 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 751/1495] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:20 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:20.784228 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0572ce2-72ba-431d-8215-94df91f8c61f None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:20.785973 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-a0572ce2-72ba-431d-8215-94df91f8c61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:20.871486 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 745/1496] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:20.949078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0572ce2-72ba-431d-8215-94df91f8c61f None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:20.960969 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a66d4-fcc1-4c70-b86a-e1a2d093e1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:20.961678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a66d4-fcc1-4c70-b86a-e1a2d093e1f7 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:20.961963 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a66d4-fcc1-4c70-b86a-e1a2d093e1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:20.962241 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a66d4-fcc1-4c70-b86a-e1a2d093e1f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:20.962899 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a66d4-fcc1-4c70-b86a-e1a2d093e1f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:20.969306 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a66d4-fcc1-4c70-b86a-e1a2d093e1f7 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:21.030058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a2430-b03e-40d2-87b4-6678a45ef6b0 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:21.033234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-036a2430-b03e-40d2-87b4-6678a45ef6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:21.108853 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 752/1497] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:20 2022] POST /identity/v3/projects => generated 407 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:21.112543 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a66d4-fcc1-4c70-b86a-e1a2d093e1f7 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:21.197739 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a2430-b03e-40d2-87b4-6678a45ef6b0 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:21.278968 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a66d4-fcc1-4c70-b86a-e1a2d093e1f7 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:21.289621 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c9a8e-d02c-4d13-860d-4edc65afb34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:21.290472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c9a8e-d02c-4d13-860d-4edc65afb34e admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:21.291653 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c9a8e-d02c-4d13-860d-4edc65afb34e admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:21.291985 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c9a8e-d02c-4d13-860d-4edc65afb34e admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:21.293070 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c9a8e-d02c-4d13-860d-4edc65afb34e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:21.295963 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c9a8e-d02c-4d13-860d-4edc65afb34e admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:21.373349 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c62a3d6-def0-44e0-8a60-1dcae9b79300 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:21.373999 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62a3d6-def0-44e0-8a60-1dcae9b79300 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:21.374262 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62a3d6-def0-44e0-8a60-1dcae9b79300 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:21.374535 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62a3d6-def0-44e0-8a60-1dcae9b79300 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:21.411331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 753/1498] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:21 2022] POST /identity/v3/users => generated 310 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:21.413619 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c62a3d6-def0-44e0-8a60-1dcae9b79300 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:21.415387 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c62a3d6-def0-44e0-8a60-1dcae9b79300 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:21.416151 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c9a8e-d02c-4d13-860d-4edc65afb34e admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:21.454189 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 746/1499] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:21.470464 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c9a8e-d02c-4d13-860d-4edc65afb34e admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:21.492999 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07994720-61d2-431b-89da-ca9e3b1bab54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:21.502532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:21.504951 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07994720-61d2-431b-89da-ca9e3b1bab54 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:21.505483 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07994720-61d2-431b-89da-ca9e3b1bab54 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:21.505683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07994720-61d2-431b-89da-ca9e3b1bab54 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:21.506433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07994720-61d2-431b-89da-ca9e3b1bab54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:21.511710 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07994720-61d2-431b-89da-ca9e3b1bab54 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:21.568841 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 754/1500] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:21.573449 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c62a3d6-def0-44e0-8a60-1dcae9b79300 service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:21.575218 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:21.591922 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c62a3d6-def0-44e0-8a60-1dcae9b79300 service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:21.605556 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edbbeb-0c66-40e1-af98-867a85a9c52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:21.607158 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edbbeb-0c66-40e1-af98-867a85a9c52d admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:21.607610 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edbbeb-0c66-40e1-af98-867a85a9c52d admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:21.608333 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edbbeb-0c66-40e1-af98-867a85a9c52d admin admin] PATH_INFO: `/v3/projects/58d521c8b35f4c86b195f70719cc453a/users/8e2e4f64c4854e849cb282b624138d7b/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:21.653531 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3edbbeb-0c66-40e1-af98-867a85a9c52d admin admin] RBAC: Authorizing `identity:create_grant(project_id=58d521c8b35f4c86b195f70719cc453a, user_id=8e2e4f64c4854e849cb282b624138d7b, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:21.656323 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3edbbeb-0c66-40e1-af98-867a85a9c52d admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:21.779904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 747/1501] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:21 2022] PUT /identity/v3/projects/58d521c8b35f4c86b195f70719cc453a/users/8e2e4f64c4854e849cb282b624138d7b/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:21.783440 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07994720-61d2-431b-89da-ca9e3b1bab54 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:21.808273 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07994720-61d2-431b-89da-ca9e3b1bab54 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:21.899202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f56bf-5bcf-49b6-ad75-95dd9c112400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:21.900307 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f56bf-5bcf-49b6-ad75-95dd9c112400 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:21.900787 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f56bf-5bcf-49b6-ad75-95dd9c112400 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:21.901266 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f56bf-5bcf-49b6-ad75-95dd9c112400 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:21.902134 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f56bf-5bcf-49b6-ad75-95dd9c112400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:21.904128 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f56bf-5bcf-49b6-ad75-95dd9c112400 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:21.944960 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 755/1502] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:21 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:21.945646 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edbbeb-0c66-40e1-af98-867a85a9c52d admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:21.961988 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edbbeb-0c66-40e1-af98-867a85a9c52d admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:21.969088 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e750dd9-1604-4b19-8690-914a8eb25d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:21.969822 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e750dd9-1604-4b19-8690-914a8eb25d41 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:21.970103 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e750dd9-1604-4b19-8690-914a8eb25d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:21.971098 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e750dd9-1604-4b19-8690-914a8eb25d41 admin admin] PATH_INFO: `/v3/projects/58d521c8b35f4c86b195f70719cc453a/users/8e2e4f64c4854e849cb282b624138d7b/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:21.973855 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e750dd9-1604-4b19-8690-914a8eb25d41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58d521c8b35f4c86b195f70719cc453a, user_id=8e2e4f64c4854e849cb282b624138d7b, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:21.980283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e750dd9-1604-4b19-8690-914a8eb25d41 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:22.032353 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28ceab-bff8-4226-8f87-1502790f6cf9 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:22.032757 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28ceab-bff8-4226-8f87-1502790f6cf9 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:22.033037 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28ceab-bff8-4226-8f87-1502790f6cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:22.037986 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 748/1503] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:21 2022] PUT /identity/v3/projects/58d521c8b35f4c86b195f70719cc453a/users/8e2e4f64c4854e849cb282b624138d7b/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:22.100315 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-ff28ceab-bff8-4226-8f87-1502790f6cf9 None None] MFA Rules not processed for user `8e2e4f64c4854e849cb282b624138d7b`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:22.101894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-ff28ceab-bff8-4226-8f87-1502790f6cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:22.218213 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 756/1504] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:22 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:22.228811 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e750dd9-1604-4b19-8690-914a8eb25d41 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:22.230975 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6e750dd9-1604-4b19-8690-914a8eb25d41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:22.353869 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e750dd9-1604-4b19-8690-914a8eb25d41 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:22.362294 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85f4a1-4d09-4c2c-b435-7f4843882b13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:22.362935 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85f4a1-4d09-4c2c-b435-7f4843882b13 service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:22.363215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85f4a1-4d09-4c2c-b435-7f4843882b13 service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:22.363519 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85f4a1-4d09-4c2c-b435-7f4843882b13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:22.379802 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df85f4a1-4d09-4c2c-b435-7f4843882b13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:22.381350 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df85f4a1-4d09-4c2c-b435-7f4843882b13 service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:22.393587 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 749/1505] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:22 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:22.674033 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:22.935191 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:23.073684 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28ceab-bff8-4226-8f87-1502790f6cf9 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:23.093407 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28ceab-bff8-4226-8f87-1502790f6cf9 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:23.112236 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2252c61-e704-4590-b826-81f936a222df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:23.125424 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2252c61-e704-4590-b826-81f936a222df service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:23.125930 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2252c61-e704-4590-b826-81f936a222df service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:23.127461 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2252c61-e704-4590-b826-81f936a222df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:23.296565 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2252c61-e704-4590-b826-81f936a222df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:23.298176 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2252c61-e704-4590-b826-81f936a222df service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:23.329971 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 757/1506] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:23 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:23.625671 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:23.727967 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85f4a1-4d09-4c2c-b435-7f4843882b13 service cinder] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:23.729585 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-df85f4a1-4d09-4c2c-b435-7f4843882b13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:23.840492 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2252c61-e704-4590-b826-81f936a222df service cinder] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:23.858408 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-d2252c61-e704-4590-b826-81f936a222df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:23.882351 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85f4a1-4d09-4c2c-b435-7f4843882b13 service cinder] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:23.890645 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2252c61-e704-4590-b826-81f936a222df service cinder] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:23.891117 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7c024-7ac9-4c6c-89e2-482dc51bb64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:23.892208 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7c024-7ac9-4c6c-89e2-482dc51bb64d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:23.892501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7c024-7ac9-4c6c-89e2-482dc51bb64d admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:23.892839 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7c024-7ac9-4c6c-89e2-482dc51bb64d admin admin] PATH_INFO: `/v3/projects/2d6cba3edebd4f0eb5343491d5102c17` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:23.894253 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f7c024-7ac9-4c6c-89e2-482dc51bb64d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d6cba3edebd4f0eb5343491d5102c17)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:23.897275 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f7c024-7ac9-4c6c-89e2-482dc51bb64d admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:23.907036 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6601d083-2265-4d0f-85ad-81d1c588e900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:23.907839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601d083-2265-4d0f-85ad-81d1c588e900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:23.912010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601d083-2265-4d0f-85ad-81d1c588e900 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:23.912010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6601d083-2265-4d0f-85ad-81d1c588e900 admin admin] PATH_INFO: `/v3/users/39c05f642d1f4ff6a77d3cff6e5527d7` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:23.912010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6601d083-2265-4d0f-85ad-81d1c588e900 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39c05f642d1f4ff6a77d3cff6e5527d7)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:23.914019 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6601d083-2265-4d0f-85ad-81d1c588e900 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:23.925841 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:24.204929 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-6601d083-2265-4d0f-85ad-81d1c588e900 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39c05f642d1f4ff6a77d3cff6e5527d7'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:24.208227 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-23f7c024-7ac9-4c6c-89e2-482dc51bb64d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d6cba3edebd4f0eb5343491d5102c17', 'request_id': 'req-23f7c024-7ac9-4c6c-89e2-482dc51bb64d'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:24.299567 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-6601d083-2265-4d0f-85ad-81d1c588e900 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39c05f642d1f4ff6a77d3cff6e5527d7'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:24.336696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-6601d083-2265-4d0f-85ad-81d1c588e900 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39c05f642d1f4ff6a77d3cff6e5527d7'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:24.337784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-6601d083-2265-4d0f-85ad-81d1c588e900 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39c05f642d1f4ff6a77d3cff6e5527d7'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:24.339177 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-6601d083-2265-4d0f-85ad-81d1c588e900 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39c05f642d1f4ff6a77d3cff6e5527d7'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:24.370023 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 758/1507] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:51:23 2022] DELETE /identity/v3/users/39c05f642d1f4ff6a77d3cff6e5527d7 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:24.371808 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8b03f-ceb3-48e7-814f-c196c1475e6c None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:24.372102 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8b03f-ceb3-48e7-814f-c196c1475e6c None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:24.372383 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8b03f-ceb3-48e7-814f-c196c1475e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:24.381120 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-23f7c024-7ac9-4c6c-89e2-482dc51bb64d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d6cba3edebd4f0eb5343491d5102c17', 'request_id': 'req-23f7c024-7ac9-4c6c-89e2-482dc51bb64d'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:24.388892 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-23f7c024-7ac9-4c6c-89e2-482dc51bb64d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d6cba3edebd4f0eb5343491d5102c17', 'request_id': 'req-23f7c024-7ac9-4c6c-89e2-482dc51bb64d'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:24.399091 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-fdf8b03f-ceb3-48e7-814f-c196c1475e6c None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:24.402481 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-fdf8b03f-ceb3-48e7-814f-c196c1475e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:24.421277 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-23f7c024-7ac9-4c6c-89e2-482dc51bb64d admin admin] The token cache is being invalidate because project 2d6cba3edebd4f0eb5343491d5102c17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:24.423483 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-23f7c024-7ac9-4c6c-89e2-482dc51bb64d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:24.425002 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-23f7c024-7ac9-4c6c-89e2-482dc51bb64d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:24.428149 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 750/1508] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:23 2022] DELETE /identity/v3/projects/2d6cba3edebd4f0eb5343491d5102c17 => generated 0 bytes in 701 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:24.432218 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7c024-7ac9-4c6c-89e2-482dc51bb64d admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:24.434306 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-23f7c024-7ac9-4c6c-89e2-482dc51bb64d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:24.502868 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 759/1509] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:51:24.562652 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7c024-7ac9-4c6c-89e2-482dc51bb64d admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:24.570147 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e788a-ba98-4986-ab7b-64f3ac70c6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:24.576730 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e788a-ba98-4986-ab7b-64f3ac70c6b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:24.577193 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e788a-ba98-4986-ab7b-64f3ac70c6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:24.577578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081e788a-ba98-4986-ab7b-64f3ac70c6b0 admin admin] PATH_INFO: `/v3/users/a7340f905c44482d9cb7d291356563c8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:24.579209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081e788a-ba98-4986-ab7b-64f3ac70c6b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7340f905c44482d9cb7d291356563c8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:24.582143 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081e788a-ba98-4986-ab7b-64f3ac70c6b0 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:24.598687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:24.617845 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8b03f-ceb3-48e7-814f-c196c1475e6c None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:24.620353 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-fdf8b03f-ceb3-48e7-814f-c196c1475e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:24.765243 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8b03f-ceb3-48e7-814f-c196c1475e6c None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:24.771432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b670e-a6a1-4f39-82be-df8c59458ce4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:24.772323 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b670e-a6a1-4f39-82be-df8c59458ce4 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:24.772645 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b670e-a6a1-4f39-82be-df8c59458ce4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:24.772969 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b670e-a6a1-4f39-82be-df8c59458ce4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:24.796716 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b670e-a6a1-4f39-82be-df8c59458ce4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:24.798378 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35b670e-a6a1-4f39-82be-df8c59458ce4 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:24.848056 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 760/1510] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:24.852146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-081e788a-ba98-4986-ab7b-64f3ac70c6b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7340f905c44482d9cb7d291356563c8'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:24.860844 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-081e788a-ba98-4986-ab7b-64f3ac70c6b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7340f905c44482d9cb7d291356563c8'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:24.958988 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-081e788a-ba98-4986-ab7b-64f3ac70c6b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7340f905c44482d9cb7d291356563c8'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:24.994637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-081e788a-ba98-4986-ab7b-64f3ac70c6b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7340f905c44482d9cb7d291356563c8'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:24.995933 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-081e788a-ba98-4986-ab7b-64f3ac70c6b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7340f905c44482d9cb7d291356563c8'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:25.022147 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 751/1511] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:51:24 2022] DELETE /identity/v3/users/a7340f905c44482d9cb7d291356563c8 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:25.762569 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b670e-a6a1-4f39-82be-df8c59458ce4 service neutron] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:25.764090 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-b35b670e-a6a1-4f39-82be-df8c59458ce4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:25.780900 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35b670e-a6a1-4f39-82be-df8c59458ce4 service neutron] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:25.800856 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:25.833376 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e788a-ba98-4986-ab7b-64f3ac70c6b0 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:25.833376 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-081e788a-ba98-4986-ab7b-64f3ac70c6b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:25.870409 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46d3ab-57c2-4d1a-86eb-ff4e1fb7ef46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:25.870944 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46d3ab-57c2-4d1a-86eb-ff4e1fb7ef46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:25.878770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46d3ab-57c2-4d1a-86eb-ff4e1fb7ef46 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:25.879513 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46d3ab-57c2-4d1a-86eb-ff4e1fb7ef46 admin admin] PATH_INFO: `/v3/users/7440bef61da145d982381f9dfc320ff6` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:25.893694 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e46d3ab-57c2-4d1a-86eb-ff4e1fb7ef46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7440bef61da145d982381f9dfc320ff6)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:25.896602 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e46d3ab-57c2-4d1a-86eb-ff4e1fb7ef46 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:25.958741 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081e788a-ba98-4986-ab7b-64f3ac70c6b0 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:25.968473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e6e7ae-ffb7-480b-b98b-e894b6c833c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:25.969377 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6e7ae-ffb7-480b-b98b-e894b6c833c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:25.972243 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6e7ae-ffb7-480b-b98b-e894b6c833c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:25.973249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e6e7ae-ffb7-480b-b98b-e894b6c833c5 admin admin] PATH_INFO: `/v3/projects/54aede97981945b297ac091584f01e9c` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:25.975672 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e6e7ae-ffb7-480b-b98b-e894b6c833c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54aede97981945b297ac091584f01e9c)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:25.983520 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e6e7ae-ffb7-480b-b98b-e894b6c833c5 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:26.100123 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-6e46d3ab-57c2-4d1a-86eb-ff4e1fb7ef46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7440bef61da145d982381f9dfc320ff6'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:26.104927 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-11e6e7ae-ffb7-480b-b98b-e894b6c833c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54aede97981945b297ac091584f01e9c', 'request_id': 'req-11e6e7ae-ffb7-480b-b98b-e894b6c833c5'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:26.231511 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-6e46d3ab-57c2-4d1a-86eb-ff4e1fb7ef46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7440bef61da145d982381f9dfc320ff6'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:26.244282 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-6e46d3ab-57c2-4d1a-86eb-ff4e1fb7ef46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7440bef61da145d982381f9dfc320ff6'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:26.244929 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-6e46d3ab-57c2-4d1a-86eb-ff4e1fb7ef46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7440bef61da145d982381f9dfc320ff6'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:26.245542 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-6e46d3ab-57c2-4d1a-86eb-ff4e1fb7ef46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7440bef61da145d982381f9dfc320ff6'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:26.258067 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 761/1512] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:51:25 2022] DELETE /identity/v3/users/7440bef61da145d982381f9dfc320ff6 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:26.258874 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46d3ab-57c2-4d1a-86eb-ff4e1fb7ef46 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:26.260199 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-6e46d3ab-57c2-4d1a-86eb-ff4e1fb7ef46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:26.321723 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46d3ab-57c2-4d1a-86eb-ff4e1fb7ef46 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:26.326327 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2258e-efb8-4f7c-8a7b-bdaf716cafe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:26.326967 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2258e-efb8-4f7c-8a7b-bdaf716cafe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:26.327245 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2258e-efb8-4f7c-8a7b-bdaf716cafe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:26.327520 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d2258e-efb8-4f7c-8a7b-bdaf716cafe6 admin admin] PATH_INFO: `/v3/projects/e2391d1f58f24d188963a0eb7cf57776` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:26.328759 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d2258e-efb8-4f7c-8a7b-bdaf716cafe6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2391d1f58f24d188963a0eb7cf57776)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:26.330732 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d2258e-efb8-4f7c-8a7b-bdaf716cafe6 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:26.362669 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-11e6e7ae-ffb7-480b-b98b-e894b6c833c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54aede97981945b297ac091584f01e9c', 'request_id': 'req-11e6e7ae-ffb7-480b-b98b-e894b6c833c5'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:26.368575 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-11e6e7ae-ffb7-480b-b98b-e894b6c833c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54aede97981945b297ac091584f01e9c', 'request_id': 'req-11e6e7ae-ffb7-480b-b98b-e894b6c833c5'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:26.380674 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-11e6e7ae-ffb7-480b-b98b-e894b6c833c5 admin admin] The token cache is being invalidate because project 54aede97981945b297ac091584f01e9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:26.380968 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-11e6e7ae-ffb7-480b-b98b-e894b6c833c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:26.381635 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-11e6e7ae-ffb7-480b-b98b-e894b6c833c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:26.386721 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 752/1513] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:25 2022] DELETE /identity/v3/projects/54aede97981945b297ac091584f01e9c => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:26.386721 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a87f5-013d-403e-9bdf-fc30f1f9b4c4 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:26.386721 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a87f5-013d-403e-9bdf-fc30f1f9b4c4 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:26.386721 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a87f5-013d-403e-9bdf-fc30f1f9b4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:26.411670 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-822a87f5-013d-403e-9bdf-fc30f1f9b4c4 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:26.413096 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-822a87f5-013d-403e-9bdf-fc30f1f9b4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:26.420874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 753/1514] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:26.421676 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a87f5-013d-403e-9bdf-fc30f1f9b4c4 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:26.428844 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-822a87f5-013d-403e-9bdf-fc30f1f9b4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:26.442726 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a87f5-013d-403e-9bdf-fc30f1f9b4c4 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:26.447567 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75be912-9a7b-4a92-aecc-5a46eba398be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:26.448397 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75be912-9a7b-4a92-aecc-5a46eba398be admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:26.448746 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75be912-9a7b-4a92-aecc-5a46eba398be admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:26.452067 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75be912-9a7b-4a92-aecc-5a46eba398be admin admin] PATH_INFO: `/v3/users/20b6e2199ac548f29828047a40b7e1a6` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:26.453316 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75be912-9a7b-4a92-aecc-5a46eba398be admin admin] RBAC: Authorizing `identity:delete_user(user_id=20b6e2199ac548f29828047a40b7e1a6)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:26.455323 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75be912-9a7b-4a92-aecc-5a46eba398be admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:26.501088 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-a2d2258e-efb8-4f7c-8a7b-bdaf716cafe6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2391d1f58f24d188963a0eb7cf57776', 'request_id': 'req-a2d2258e-efb8-4f7c-8a7b-bdaf716cafe6'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:26.590288 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-a2d2258e-efb8-4f7c-8a7b-bdaf716cafe6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2391d1f58f24d188963a0eb7cf57776', 'request_id': 'req-a2d2258e-efb8-4f7c-8a7b-bdaf716cafe6'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:26.590566 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:26.596747 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-a2d2258e-efb8-4f7c-8a7b-bdaf716cafe6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2391d1f58f24d188963a0eb7cf57776', 'request_id': 'req-a2d2258e-efb8-4f7c-8a7b-bdaf716cafe6'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:26.608617 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-a2d2258e-efb8-4f7c-8a7b-bdaf716cafe6 admin admin] The token cache is being invalidate because project e2391d1f58f24d188963a0eb7cf57776 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:26.608937 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-a2d2258e-efb8-4f7c-8a7b-bdaf716cafe6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:26.609600 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-a2d2258e-efb8-4f7c-8a7b-bdaf716cafe6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:26.611197 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 762/1515] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:26 2022] DELETE /identity/v3/projects/e2391d1f58f24d188963a0eb7cf57776 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:26.612002 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2258e-efb8-4f7c-8a7b-bdaf716cafe6 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:26.613734 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-a2d2258e-efb8-4f7c-8a7b-bdaf716cafe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:26.667836 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d2258e-efb8-4f7c-8a7b-bdaf716cafe6 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:26.668271 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-a75be912-9a7b-4a92-aecc-5a46eba398be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20b6e2199ac548f29828047a40b7e1a6'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:26.668924 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-a75be912-9a7b-4a92-aecc-5a46eba398be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20b6e2199ac548f29828047a40b7e1a6'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:26.671485 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde2334-1640-4cf9-973d-75962bd7c6a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:26.672118 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde2334-1640-4cf9-973d-75962bd7c6a0 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:26.672396 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde2334-1640-4cf9-973d-75962bd7c6a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:26.672690 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde2334-1640-4cf9-973d-75962bd7c6a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:26.674367 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-0fde2334-1640-4cf9-973d-75962bd7c6a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:26.728911 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-a75be912-9a7b-4a92-aecc-5a46eba398be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20b6e2199ac548f29828047a40b7e1a6'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:26.739138 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde2334-1640-4cf9-973d-75962bd7c6a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:26.740726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde2334-1640-4cf9-973d-75962bd7c6a0 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:26.742051 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-a75be912-9a7b-4a92-aecc-5a46eba398be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20b6e2199ac548f29828047a40b7e1a6'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:26.742987 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-a75be912-9a7b-4a92-aecc-5a46eba398be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20b6e2199ac548f29828047a40b7e1a6'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:26.750144 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 763/1516] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:26.751405 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825861b-6730-46b2-b13a-358dc9838266 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:26.751662 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825861b-6730-46b2-b13a-358dc9838266 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:26.751930 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825861b-6730-46b2-b13a-358dc9838266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:26.757428 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 754/1517] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:51:26 2022] DELETE /identity/v3/users/20b6e2199ac548f29828047a40b7e1a6 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:26.780652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-5825861b-6730-46b2-b13a-358dc9838266 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:26.782129 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-5825861b-6730-46b2-b13a-358dc9838266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:26.854040 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 764/1518] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:26.867417 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc34c11-f158-4fed-b65b-57d2ac019ca1 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:26.867417 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc34c11-f158-4fed-b65b-57d2ac019ca1 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:26.867417 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc34c11-f158-4fed-b65b-57d2ac019ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:26.885485 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-cbc34c11-f158-4fed-b65b-57d2ac019ca1 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:26.889078 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-cbc34c11-f158-4fed-b65b-57d2ac019ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:26.894585 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 755/1519] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:26.898914 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825861b-6730-46b2-b13a-358dc9838266 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:26.913645 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825861b-6730-46b2-b13a-358dc9838266 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:26.918349 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec038c-f28a-4d03-b454-51beb1d37491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:26.918964 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec038c-f28a-4d03-b454-51beb1d37491 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:26.919245 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec038c-f28a-4d03-b454-51beb1d37491 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:26.919515 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec038c-f28a-4d03-b454-51beb1d37491 admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:26.920190 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec038c-f28a-4d03-b454-51beb1d37491 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:51:26.920493 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec038c-f28a-4d03-b454-51beb1d37491 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:26.922065 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec038c-f28a-4d03-b454-51beb1d37491 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:26.934652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 765/1520] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:51:26.942197 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc34c11-f158-4fed-b65b-57d2ac019ca1 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:26.951040 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc34c11-f158-4fed-b65b-57d2ac019ca1 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:26.957823 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a60415-154e-4b35-95aa-8990419fc2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:26.958443 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a60415-154e-4b35-95aa-8990419fc2e4 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:26.958898 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a60415-154e-4b35-95aa-8990419fc2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:26.959314 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a60415-154e-4b35-95aa-8990419fc2e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:26.960452 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a60415-154e-4b35-95aa-8990419fc2e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:26.962745 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a60415-154e-4b35-95aa-8990419fc2e4 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:27.056052 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 756/1521] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:26 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:27.056885 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec038c-f28a-4d03-b454-51beb1d37491 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:27.072770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec038c-f28a-4d03-b454-51beb1d37491 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:27.137890 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232a8f9-8e6b-4150-9709-07a6c0ed9ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:27.138635 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232a8f9-8e6b-4150-9709-07a6c0ed9ae0 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:27.139018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232a8f9-8e6b-4150-9709-07a6c0ed9ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:27.139422 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f232a8f9-8e6b-4150-9709-07a6c0ed9ae0 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:27.140207 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f232a8f9-8e6b-4150-9709-07a6c0ed9ae0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:27.142222 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f232a8f9-8e6b-4150-9709-07a6c0ed9ae0 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:27.219996 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 766/1522] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:27 2022] POST /identity/v3/users => generated 291 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:51:27.221190 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a60415-154e-4b35-95aa-8990419fc2e4 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:27.233188 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a60415-154e-4b35-95aa-8990419fc2e4 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:27.238658 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b78628-1008-49cd-ba1d-c4259df375ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:27.239309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b78628-1008-49cd-ba1d-c4259df375ff admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:27.239584 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b78628-1008-49cd-ba1d-c4259df375ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:27.239998 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b78628-1008-49cd-ba1d-c4259df375ff admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:27.240662 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b78628-1008-49cd-ba1d-c4259df375ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:27.242249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b78628-1008-49cd-ba1d-c4259df375ff admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:27.256336 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 757/1523] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:27.258019 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232a8f9-8e6b-4150-9709-07a6c0ed9ae0 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:27.268093 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f232a8f9-8e6b-4150-9709-07a6c0ed9ae0 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:27.273459 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39c681-6631-4cc8-abe5-69f4d35efd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:27.274054 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39c681-6631-4cc8-abe5-69f4d35efd4b admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:27.274335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39c681-6631-4cc8-abe5-69f4d35efd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:27.274610 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39c681-6631-4cc8-abe5-69f4d35efd4b admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:27.275395 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39c681-6631-4cc8-abe5-69f4d35efd4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:27.277016 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc39c681-6631-4cc8-abe5-69f4d35efd4b admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:27.289499 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 767/1524] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:27.292523 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b78628-1008-49cd-ba1d-c4259df375ff admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:27.314297 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b78628-1008-49cd-ba1d-c4259df375ff admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:27.314297 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f5221-0bff-43fb-b78c-db632d06900e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:27.314297 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f5221-0bff-43fb-b78c-db632d06900e admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:27.314297 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f5221-0bff-43fb-b78c-db632d06900e admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:27.314630 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f5221-0bff-43fb-b78c-db632d06900e admin admin] PATH_INFO: `/v3/projects/1644e7940fa1423f89ad3654b27e905e/users/534c283d68754261a521ff28325a5165/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:27.324889 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827f5221-0bff-43fb-b78c-db632d06900e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1644e7940fa1423f89ad3654b27e905e, user_id=534c283d68754261a521ff28325a5165, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:27.327605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827f5221-0bff-43fb-b78c-db632d06900e admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:27.351355 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39c681-6631-4cc8-abe5-69f4d35efd4b admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:27.353140 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-bc39c681-6631-4cc8-abe5-69f4d35efd4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:27.367798 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39c681-6631-4cc8-abe5-69f4d35efd4b admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:27.373136 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c9c6aa-56ed-467c-89f0-6ac2f6c43dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:27.373778 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9c6aa-56ed-467c-89f0-6ac2f6c43dd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:27.374063 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9c6aa-56ed-467c-89f0-6ac2f6c43dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:27.378787 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9c6aa-56ed-467c-89f0-6ac2f6c43dd0 admin admin] PATH_INFO: `/v3/projects/6d557d2ff4424da2916a9aeddc602f48` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:27.378787 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c9c6aa-56ed-467c-89f0-6ac2f6c43dd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d557d2ff4424da2916a9aeddc602f48)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:27.378787 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c9c6aa-56ed-467c-89f0-6ac2f6c43dd0 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:27.437254 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 758/1525] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:27 2022] PUT /identity/v3/projects/1644e7940fa1423f89ad3654b27e905e/users/534c283d68754261a521ff28325a5165/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:27.438134 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f5221-0bff-43fb-b78c-db632d06900e admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:27.440054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-827f5221-0bff-43fb-b78c-db632d06900e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:27.447904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:27.512335 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827f5221-0bff-43fb-b78c-db632d06900e admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:27.518484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-86c9c6aa-56ed-467c-89f0-6ac2f6c43dd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d557d2ff4424da2916a9aeddc602f48', 'request_id': 'req-86c9c6aa-56ed-467c-89f0-6ac2f6c43dd0'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:27.579998 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-86c9c6aa-56ed-467c-89f0-6ac2f6c43dd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d557d2ff4424da2916a9aeddc602f48', 'request_id': 'req-86c9c6aa-56ed-467c-89f0-6ac2f6c43dd0'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:27.583330 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb785fa-6f1b-4f24-b137-50714455abaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:27.584336 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb785fa-6f1b-4f24-b137-50714455abaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:27.584918 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb785fa-6f1b-4f24-b137-50714455abaa admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:27.585345 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb785fa-6f1b-4f24-b137-50714455abaa admin admin] PATH_INFO: `/v3/projects/c80adecb90e44f7886c97b41b26f7f8f` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:27.587109 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb785fa-6f1b-4f24-b137-50714455abaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=c80adecb90e44f7886c97b41b26f7f8f)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:27.589253 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb785fa-6f1b-4f24-b137-50714455abaa admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:27.590226 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-86c9c6aa-56ed-467c-89f0-6ac2f6c43dd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d557d2ff4424da2916a9aeddc602f48', 'request_id': 'req-86c9c6aa-56ed-467c-89f0-6ac2f6c43dd0'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:27.604120 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-86c9c6aa-56ed-467c-89f0-6ac2f6c43dd0 admin admin] The token cache is being invalidate because project 6d557d2ff4424da2916a9aeddc602f48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:27.604410 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-86c9c6aa-56ed-467c-89f0-6ac2f6c43dd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:27.605314 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-86c9c6aa-56ed-467c-89f0-6ac2f6c43dd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:27.607238 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 768/1526] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:27 2022] DELETE /identity/v3/projects/6d557d2ff4424da2916a9aeddc602f48 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:51:27.608590 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332f6e4-6ba4-4366-83e0-4a08f8798b04 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:27.608872 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332f6e4-6ba4-4366-83e0-4a08f8798b04 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:27.609143 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b332f6e4-6ba4-4366-83e0-4a08f8798b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:27.652297 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-b332f6e4-6ba4-4366-83e0-4a08f8798b04 None None] MFA Rules not processed for user `534c283d68754261a521ff28325a5165`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:27.653524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-b332f6e4-6ba4-4366-83e0-4a08f8798b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:27.725740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-beb785fa-6f1b-4f24-b137-50714455abaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c80adecb90e44f7886c97b41b26f7f8f', 'request_id': 'req-beb785fa-6f1b-4f24-b137-50714455abaa'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:27.747837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 769/1527] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:27.749014 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b332f6e4-6ba4-4366-83e0-4a08f8798b04 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:27.750904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-b332f6e4-6ba4-4366-83e0-4a08f8798b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:27.776014 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b332f6e4-6ba4-4366-83e0-4a08f8798b04 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:27.791936 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:27.814344 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-beb785fa-6f1b-4f24-b137-50714455abaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c80adecb90e44f7886c97b41b26f7f8f', 'request_id': 'req-beb785fa-6f1b-4f24-b137-50714455abaa'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:27.821224 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-beb785fa-6f1b-4f24-b137-50714455abaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c80adecb90e44f7886c97b41b26f7f8f', 'request_id': 'req-beb785fa-6f1b-4f24-b137-50714455abaa'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:27.821800 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced6e01-3b2f-4dcf-b6bf-bac04e9354e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:27.822441 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced6e01-3b2f-4dcf-b6bf-bac04e9354e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:27.822955 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced6e01-3b2f-4dcf-b6bf-bac04e9354e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:27.824220 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced6e01-3b2f-4dcf-b6bf-bac04e9354e7 admin admin] PATH_INFO: `/v3/users/e9979a5a177e43b88cdf52f7e15ffb49` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:27.826010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced6e01-3b2f-4dcf-b6bf-bac04e9354e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9979a5a177e43b88cdf52f7e15ffb49)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:27.828214 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced6e01-3b2f-4dcf-b6bf-bac04e9354e7 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:27.833529 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-beb785fa-6f1b-4f24-b137-50714455abaa admin admin] The token cache is being invalidate because project c80adecb90e44f7886c97b41b26f7f8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:27.833818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-beb785fa-6f1b-4f24-b137-50714455abaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:27.834479 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-beb785fa-6f1b-4f24-b137-50714455abaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:27.836024 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 759/1528] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:27 2022] DELETE /identity/v3/projects/c80adecb90e44f7886c97b41b26f7f8f => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:27.836769 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb785fa-6f1b-4f24-b137-50714455abaa admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:27.838047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-beb785fa-6f1b-4f24-b137-50714455abaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:27.850742 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb785fa-6f1b-4f24-b137-50714455abaa admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:27.856246 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd6abc-ef51-488e-9571-b059de266d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:27.856385 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd6abc-ef51-488e-9571-b059de266d0e admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:27.856613 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd6abc-ef51-488e-9571-b059de266d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:27.856891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd6abc-ef51-488e-9571-b059de266d0e admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:27.857525 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cd6abc-ef51-488e-9571-b059de266d0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:27.859430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cd6abc-ef51-488e-9571-b059de266d0e admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:27.990965 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 760/1529] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:27 2022] POST /identity/v3/projects => generated 383 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:27.992270 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b6672-19e8-4b30-88a8-eb938ea30983 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:27.992538 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b6672-19e8-4b30-88a8-eb938ea30983 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:27.992811 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b6672-19e8-4b30-88a8-eb938ea30983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:28.012016 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-b79b6672-19e8-4b30-88a8-eb938ea30983 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:28.016343 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-b79b6672-19e8-4b30-88a8-eb938ea30983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:28.021677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-8ced6e01-3b2f-4dcf-b6bf-bac04e9354e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9979a5a177e43b88cdf52f7e15ffb49'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:28.079537 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 761/1530] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:28.079537 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b6672-19e8-4b30-88a8-eb938ea30983 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:28.081868 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b6672-19e8-4b30-88a8-eb938ea30983 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:28.086445 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd64cb-5553-4a23-922b-23977c80c472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:28.087043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd64cb-5553-4a23-922b-23977c80c472 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:28.087324 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd64cb-5553-4a23-922b-23977c80c472 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:28.087646 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd64cb-5553-4a23-922b-23977c80c472 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:28.088309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fd64cb-5553-4a23-922b-23977c80c472 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:28.090197 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fd64cb-5553-4a23-922b-23977c80c472 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:28.095859 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-8ced6e01-3b2f-4dcf-b6bf-bac04e9354e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9979a5a177e43b88cdf52f7e15ffb49'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:28.106193 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-8ced6e01-3b2f-4dcf-b6bf-bac04e9354e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9979a5a177e43b88cdf52f7e15ffb49'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:28.107506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-8ced6e01-3b2f-4dcf-b6bf-bac04e9354e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9979a5a177e43b88cdf52f7e15ffb49'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:28.107670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-8ced6e01-3b2f-4dcf-b6bf-bac04e9354e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9979a5a177e43b88cdf52f7e15ffb49'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:28.120317 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 770/1531] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:51:27 2022] DELETE /identity/v3/users/e9979a5a177e43b88cdf52f7e15ffb49 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:28.121663 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ea2a2-079a-4324-86f9-cb6e44e65f8f None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:28.121952 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ea2a2-079a-4324-86f9-cb6e44e65f8f None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:28.122226 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ea2a2-079a-4324-86f9-cb6e44e65f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:28.143973 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-7c6ea2a2-079a-4324-86f9-cb6e44e65f8f None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:28.145260 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-7c6ea2a2-079a-4324-86f9-cb6e44e65f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:28.195688 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 762/1532] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:28 2022] POST /identity/v3/users => generated 298 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:28.199540 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd64cb-5553-4a23-922b-23977c80c472 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:28.202297 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-52fd64cb-5553-4a23-922b-23977c80c472 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:28.218496 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 771/1533] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:28.221360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd64cb-5553-4a23-922b-23977c80c472 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:28.223161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ea2a2-079a-4324-86f9-cb6e44e65f8f None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:28.226659 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e7f76-c1ef-4db5-8e8e-6828276368a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:28.227579 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e7f76-c1ef-4db5-8e8e-6828276368a9 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:28.227991 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e7f76-c1ef-4db5-8e8e-6828276368a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:28.228360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e7f76-c1ef-4db5-8e8e-6828276368a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:28.229621 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e7f76-c1ef-4db5-8e8e-6828276368a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:28.231515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e7f76-c1ef-4db5-8e8e-6828276368a9 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:28.232230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ea2a2-079a-4324-86f9-cb6e44e65f8f None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:28.238344 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff24e5-a815-4537-82a1-fe0a663cf2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:28.239114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff24e5-a815-4537-82a1-fe0a663cf2e1 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:28.239576 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff24e5-a815-4537-82a1-fe0a663cf2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:28.239877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff24e5-a815-4537-82a1-fe0a663cf2e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:28.244914 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff24e5-a815-4537-82a1-fe0a663cf2e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:51:28.244914 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff24e5-a815-4537-82a1-fe0a663cf2e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:28.244914 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff24e5-a815-4537-82a1-fe0a663cf2e1 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:28.245155 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 763/1534] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:51:28.248612 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e7f76-c1ef-4db5-8e8e-6828276368a9 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:28.258927 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 772/1535] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:51:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:51:28.259957 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff24e5-a815-4537-82a1-fe0a663cf2e1 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:28.260085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e7f76-c1ef-4db5-8e8e-6828276368a9 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:28.266226 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb5272-aa3d-4379-8443-4ab76c2522a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:28.267036 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb5272-aa3d-4379-8443-4ab76c2522a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:28.267547 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb5272-aa3d-4379-8443-4ab76c2522a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:28.267865 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb5272-aa3d-4379-8443-4ab76c2522a2 admin admin] PATH_INFO: `/v3/projects/9df46be471214add89a6bb3651530cc5/users/b10b44d2f74043c8bdd5b96912ccd54d/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:28.271845 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff24e5-a815-4537-82a1-fe0a663cf2e1 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:28.276267 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3065464-d25b-45a2-8a06-deb4e20a58b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:28.279576 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3065464-d25b-45a2-8a06-deb4e20a58b9 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:28.279576 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3065464-d25b-45a2-8a06-deb4e20a58b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:28.279576 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3065464-d25b-45a2-8a06-deb4e20a58b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:28.279576 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3065464-d25b-45a2-8a06-deb4e20a58b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:28.280456 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3065464-d25b-45a2-8a06-deb4e20a58b9 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:28.283800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb5272-aa3d-4379-8443-4ab76c2522a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9df46be471214add89a6bb3651530cc5, user_id=b10b44d2f74043c8bdd5b96912ccd54d, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:28.286567 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fb5272-aa3d-4379-8443-4ab76c2522a2 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:28.403200 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 764/1536] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:28 2022] PUT /identity/v3/projects/9df46be471214add89a6bb3651530cc5/users/b10b44d2f74043c8bdd5b96912ccd54d/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:51:28.409652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:28.411626 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb5272-aa3d-4379-8443-4ab76c2522a2 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:28.419717 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 773/1537] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:28 2022] POST /identity/v3/projects => generated 379 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:51:28.423527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3065464-d25b-45a2-8a06-deb4e20a58b9 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:28.502761 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3065464-d25b-45a2-8a06-deb4e20a58b9 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:28.503588 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb22b63-3ef7-4ab4-845e-d690fa59e11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:28.504434 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb22b63-3ef7-4ab4-845e-d690fa59e11e admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:28.504784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb22b63-3ef7-4ab4-845e-d690fa59e11e admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:28.505130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb22b63-3ef7-4ab4-845e-d690fa59e11e admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:28.505842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb22b63-3ef7-4ab4-845e-d690fa59e11e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:28.506989 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fb5272-aa3d-4379-8443-4ab76c2522a2 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:28.508013 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb22b63-3ef7-4ab4-845e-d690fa59e11e admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:28.512105 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe373b-8d4d-4a56-8bca-088a23c31800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:28.512815 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe373b-8d4d-4a56-8bca-088a23c31800 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:28.513195 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe373b-8d4d-4a56-8bca-088a23c31800 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:28.513572 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe373b-8d4d-4a56-8bca-088a23c31800 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:28.514440 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe373b-8d4d-4a56-8bca-088a23c31800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:28.516150 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe373b-8d4d-4a56-8bca-088a23c31800 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:28.526848 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 765/1538] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:28.533415 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe373b-8d4d-4a56-8bca-088a23c31800 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:28.543857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe373b-8d4d-4a56-8bca-088a23c31800 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:28.548507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92923e-3ab6-4fbe-ae02-5440dc673d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:28.549147 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92923e-3ab6-4fbe-ae02-5440dc673d63 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:28.549434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92923e-3ab6-4fbe-ae02-5440dc673d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:28.549715 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92923e-3ab6-4fbe-ae02-5440dc673d63 admin admin] PATH_INFO: `/v3/projects/9df46be471214add89a6bb3651530cc5/users/b10b44d2f74043c8bdd5b96912ccd54d/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:28.552342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92923e-3ab6-4fbe-ae02-5440dc673d63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9df46be471214add89a6bb3651530cc5, user_id=b10b44d2f74043c8bdd5b96912ccd54d, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:28.555001 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92923e-3ab6-4fbe-ae02-5440dc673d63 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:28.565027 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 766/1539] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:28 2022] PUT /identity/v3/projects/9df46be471214add89a6bb3651530cc5/users/b10b44d2f74043c8bdd5b96912ccd54d/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:51:28.574971 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99512b-c0b6-4f71-9d4a-bf247c131829 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:28.575266 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99512b-c0b6-4f71-9d4a-bf247c131829 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:28.575549 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99512b-c0b6-4f71-9d4a-bf247c131829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:28.589093 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 774/1540] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:28 2022] POST /identity/v3/users => generated 290 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:51:28.589093 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb22b63-3ef7-4ab4-845e-d690fa59e11e admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:28.592527 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:28.593749 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-ffb22b63-3ef7-4ab4-845e-d690fa59e11e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:28.617246 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-fe99512b-c0b6-4f71-9d4a-bf247c131829 None None] MFA Rules not processed for user `b10b44d2f74043c8bdd5b96912ccd54d`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:28.617246 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-fe99512b-c0b6-4f71-9d4a-bf247c131829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:28.619012 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb22b63-3ef7-4ab4-845e-d690fa59e11e admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:28.675976 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1266d285-0e88-4063-8310-2c4681594428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:28.676620 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266d285-0e88-4063-8310-2c4681594428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:28.676902 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266d285-0e88-4063-8310-2c4681594428 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:28.677177 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266d285-0e88-4063-8310-2c4681594428 admin admin] PATH_INFO: `/v3/projects/e0374528600d4afcb70c006e9fb44bce` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:28.683958 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1266d285-0e88-4063-8310-2c4681594428 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0374528600d4afcb70c006e9fb44bce)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:28.685877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1266d285-0e88-4063-8310-2c4681594428 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:28.688606 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 767/1541] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:51:28.689449 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99512b-c0b6-4f71-9d4a-bf247c131829 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:28.707722 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99512b-c0b6-4f71-9d4a-bf247c131829 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:28.712616 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a2c128-f92a-450b-b278-f75b257da835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:28.713351 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2c128-f92a-450b-b278-f75b257da835 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:28.713634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2c128-f92a-450b-b278-f75b257da835 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:28.713906 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a2c128-f92a-450b-b278-f75b257da835 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:28.714550 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a2c128-f92a-450b-b278-f75b257da835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:28.716237 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a2c128-f92a-450b-b278-f75b257da835 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:28.726982 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 768/1542] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:28.727982 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a2c128-f92a-450b-b278-f75b257da835 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:28.729583 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-d5a2c128-f92a-450b-b278-f75b257da835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:28.790148 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a2c128-f92a-450b-b278-f75b257da835 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:28.807631 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-1266d285-0e88-4063-8310-2c4681594428 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0374528600d4afcb70c006e9fb44bce', 'request_id': 'req-1266d285-0e88-4063-8310-2c4681594428'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:28.850752 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb9ec3-5fde-4d66-9f6c-1fc8090c2a2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:28.851368 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb9ec3-5fde-4d66-9f6c-1fc8090c2a2e service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:28.851646 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb9ec3-5fde-4d66-9f6c-1fc8090c2a2e service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:28.852134 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb9ec3-5fde-4d66-9f6c-1fc8090c2a2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:28.883251 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-1266d285-0e88-4063-8310-2c4681594428 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0374528600d4afcb70c006e9fb44bce', 'request_id': 'req-1266d285-0e88-4063-8310-2c4681594428'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:28.891245 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-1266d285-0e88-4063-8310-2c4681594428 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0374528600d4afcb70c006e9fb44bce', 'request_id': 'req-1266d285-0e88-4063-8310-2c4681594428'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:28.907244 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-1266d285-0e88-4063-8310-2c4681594428 admin admin] The token cache is being invalidate because project e0374528600d4afcb70c006e9fb44bce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:28.907542 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-1266d285-0e88-4063-8310-2c4681594428 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:28.908347 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-1266d285-0e88-4063-8310-2c4681594428 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:28.911598 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 775/1543] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:28 2022] DELETE /identity/v3/projects/e0374528600d4afcb70c006e9fb44bce => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:28.928069 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1266d285-0e88-4063-8310-2c4681594428 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:28.928069 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-1266d285-0e88-4063-8310-2c4681594428 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:28.945107 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb9ec3-5fde-4d66-9f6c-1fc8090c2a2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:28.947198 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb9ec3-5fde-4d66-9f6c-1fc8090c2a2e service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:28.959674 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 769/1544] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:28 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:28.961005 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26575d9-a77d-4e18-b60f-a3c203236405 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:28.961387 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26575d9-a77d-4e18-b60f-a3c203236405 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:28.961621 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26575d9-a77d-4e18-b60f-a3c203236405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:28.982619 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-b26575d9-a77d-4e18-b60f-a3c203236405 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:28.987082 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:28.991768 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1266d285-0e88-4063-8310-2c4681594428 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:28.995848 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-b26575d9-a77d-4e18-b60f-a3c203236405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:28.996977 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fac1cd-8cc6-4fbc-97fc-772a0c3ee4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:28.997712 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fac1cd-8cc6-4fbc-97fc-772a0c3ee4f7 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:28.998069 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fac1cd-8cc6-4fbc-97fc-772a0c3ee4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:28.998427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fac1cd-8cc6-4fbc-97fc-772a0c3ee4f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:28.999138 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fac1cd-8cc6-4fbc-97fc-772a0c3ee4f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:29.000888 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fac1cd-8cc6-4fbc-97fc-772a0c3ee4f7 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:29.009131 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 770/1545] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:29.017293 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 776/1546] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:28 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:29.017518 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9ef2a-6dba-4a23-8372-f57b64765a07 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:29.017518 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9ef2a-6dba-4a23-8372-f57b64765a07 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:29.017518 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9ef2a-6dba-4a23-8372-f57b64765a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:29.018956 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fac1cd-8cc6-4fbc-97fc-772a0c3ee4f7 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:29.029919 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fac1cd-8cc6-4fbc-97fc-772a0c3ee4f7 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:29.035718 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73edece-a7fa-442b-a3bd-4684405636a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:29.044046 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73edece-a7fa-442b-a3bd-4684405636a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:29.044046 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73edece-a7fa-442b-a3bd-4684405636a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:29.044046 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73edece-a7fa-442b-a3bd-4684405636a5 admin admin] PATH_INFO: `/v3/projects/e7cab2a3bf1849e9845c27da6f0c4c90/users/e292f77e8d704772b9becd5171fbec21/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:29.044320 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-4ef9ef2a-6dba-4a23-8372-f57b64765a07 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:29.044320 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-4ef9ef2a-6dba-4a23-8372-f57b64765a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:29.051241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 771/1547] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:51:29.055461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef9ef2a-6dba-4a23-8372-f57b64765a07 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:29.055688 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73edece-a7fa-442b-a3bd-4684405636a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7cab2a3bf1849e9845c27da6f0c4c90, user_id=e292f77e8d704772b9becd5171fbec21, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:29.058403 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73edece-a7fa-442b-a3bd-4684405636a5 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:29.064882 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef9ef2a-6dba-4a23-8372-f57b64765a07 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:29.067802 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:29.073282 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10322630-5471-4803-badc-5003f05e60e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:29.073915 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10322630-5471-4803-badc-5003f05e60e1 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:29.074193 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10322630-5471-4803-badc-5003f05e60e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:29.074465 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10322630-5471-4803-badc-5003f05e60e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:29.075127 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10322630-5471-4803-badc-5003f05e60e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:51:29.075439 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10322630-5471-4803-badc-5003f05e60e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:29.077098 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10322630-5471-4803-badc-5003f05e60e1 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:29.089044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 772/1548] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:51:29.093463 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10322630-5471-4803-badc-5003f05e60e1 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:29.103494 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10322630-5471-4803-badc-5003f05e60e1 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:29.107583 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d198f-8b90-40e6-b007-4a294d8ae77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:29.108218 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d198f-8b90-40e6-b007-4a294d8ae77b admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:29.108493 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d198f-8b90-40e6-b007-4a294d8ae77b admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:29.108763 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d198f-8b90-40e6-b007-4a294d8ae77b admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:29.109384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d198f-8b90-40e6-b007-4a294d8ae77b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:29.111296 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d198f-8b90-40e6-b007-4a294d8ae77b admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:29.158081 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 777/1549] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:29 2022] PUT /identity/v3/projects/e7cab2a3bf1849e9845c27da6f0c4c90/users/e292f77e8d704772b9becd5171fbec21/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:51:29.159012 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73edece-a7fa-442b-a3bd-4684405636a5 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:29.163770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-b73edece-a7fa-442b-a3bd-4684405636a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:29.215978 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 773/1550] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:29 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:51:29.216804 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73edece-a7fa-442b-a3bd-4684405636a5 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:29.217785 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17943ec9-5f40-43b5-b832-bbd12ef08d36 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:29.218242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17943ec9-5f40-43b5-b832-bbd12ef08d36 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:29.218523 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17943ec9-5f40-43b5-b832-bbd12ef08d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:29.246893 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-17943ec9-5f40-43b5-b832-bbd12ef08d36 None None] MFA Rules not processed for user `e292f77e8d704772b9becd5171fbec21`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:29.248144 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-17943ec9-5f40-43b5-b832-bbd12ef08d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:29.266674 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64af8bb-757d-4593-9959-1cef7ce1607f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:29.267274 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64af8bb-757d-4593-9959-1cef7ce1607f service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:29.267549 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64af8bb-757d-4593-9959-1cef7ce1607f service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:29.267849 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64af8bb-757d-4593-9959-1cef7ce1607f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:29.269581 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-e64af8bb-757d-4593-9959-1cef7ce1607f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:29.313244 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 774/1551] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:29.314183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17943ec9-5f40-43b5-b832-bbd12ef08d36 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:29.330168 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64af8bb-757d-4593-9959-1cef7ce1607f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:29.332053 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64af8bb-757d-4593-9959-1cef7ce1607f service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:29.341583 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 778/1552] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:29 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:29.342460 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64af8bb-757d-4593-9959-1cef7ce1607f service cinder] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:29.381381 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:29.397732 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17943ec9-5f40-43b5-b832-bbd12ef08d36 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:29.403223 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7388d5-7794-4f6c-9e9e-f75b649904fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:29.404014 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7388d5-7794-4f6c-9e9e-f75b649904fe admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:29.404308 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7388d5-7794-4f6c-9e9e-f75b649904fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:29.404591 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7388d5-7794-4f6c-9e9e-f75b649904fe admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:29.405247 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7388d5-7794-4f6c-9e9e-f75b649904fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:29.407249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7388d5-7794-4f6c-9e9e-f75b649904fe admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:29.419435 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:29.433712 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64af8bb-757d-4593-9959-1cef7ce1607f service cinder] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:29.438323 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d294b6-4356-4aec-895a-974d0864e61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:29.438925 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d294b6-4356-4aec-895a-974d0864e61b admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:29.439201 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d294b6-4356-4aec-895a-974d0864e61b admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:29.439483 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d294b6-4356-4aec-895a-974d0864e61b admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:29.440290 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d294b6-4356-4aec-895a-974d0864e61b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:29.442247 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d294b6-4356-4aec-895a-974d0864e61b admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:29.524004 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 779/1553] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:29 2022] POST /identity/v3/projects => generated 381 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:29.525040 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d294b6-4356-4aec-895a-974d0864e61b admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:29.529207 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 775/1554] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:29 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:29.533138 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7388d5-7794-4f6c-9e9e-f75b649904fe admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:29.553547 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7388d5-7794-4f6c-9e9e-f75b649904fe admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:29.614375 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d294b6-4356-4aec-895a-974d0864e61b admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:29.621232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133cb51-0b02-4ee7-b0ff-b4bd88c35358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:29.621872 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133cb51-0b02-4ee7-b0ff-b4bd88c35358 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:29.622131 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133cb51-0b02-4ee7-b0ff-b4bd88c35358 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:29.623583 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b133cb51-0b02-4ee7-b0ff-b4bd88c35358 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:29.624347 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133cb51-0b02-4ee7-b0ff-b4bd88c35358 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:29.627209 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b133cb51-0b02-4ee7-b0ff-b4bd88c35358 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:29.632988 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b0756-668a-4e58-8a5c-b76de15d8f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:29.633879 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b0756-668a-4e58-8a5c-b76de15d8f55 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:29.634269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b0756-668a-4e58-8a5c-b76de15d8f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:29.634772 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b0756-668a-4e58-8a5c-b76de15d8f55 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:29.636480 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b0756-668a-4e58-8a5c-b76de15d8f55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:29.638177 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b0756-668a-4e58-8a5c-b76de15d8f55 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:29.655212 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 776/1555] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:29.659283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b0756-668a-4e58-8a5c-b76de15d8f55 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:29.670196 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b0756-668a-4e58-8a5c-b76de15d8f55 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:29.675466 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd8831-2da4-4958-8970-ad9c63362bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:29.676217 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd8831-2da4-4958-8970-ad9c63362bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:29.676566 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd8831-2da4-4958-8970-ad9c63362bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:29.676927 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd8831-2da4-4958-8970-ad9c63362bf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:29.677637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd8831-2da4-4958-8970-ad9c63362bf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:29.679395 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd8831-2da4-4958-8970-ad9c63362bf9 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:29.689807 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 777/1556] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:51:29.693923 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd8831-2da4-4958-8970-ad9c63362bf9 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:29.704593 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd8831-2da4-4958-8970-ad9c63362bf9 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:29.711929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390bb7b-4c01-47e2-aab9-81490e85f566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:29.712610 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390bb7b-4c01-47e2-aab9-81490e85f566 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:29.712892 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390bb7b-4c01-47e2-aab9-81490e85f566 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:29.713290 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390bb7b-4c01-47e2-aab9-81490e85f566 admin admin] PATH_INFO: `/v3/projects/e1bf4b9a5bab4c6ea3c9c24eb93a909f/users/0ea7d079331240e2858d5f546490c6f5/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:29.731678 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7390bb7b-4c01-47e2-aab9-81490e85f566 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1bf4b9a5bab4c6ea3c9c24eb93a909f, user_id=0ea7d079331240e2858d5f546490c6f5, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:29.735761 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 780/1557] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:29 2022] POST /identity/v3/users => generated 297 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:29.735895 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7390bb7b-4c01-47e2-aab9-81490e85f566 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:29.737243 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133cb51-0b02-4ee7-b0ff-b4bd88c35358 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:29.739503 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:29.752213 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b133cb51-0b02-4ee7-b0ff-b4bd88c35358 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:29.759649 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73b30a-f0ff-418d-ac95-d6450b941e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:29.759649 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73b30a-f0ff-418d-ac95-d6450b941e8a admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:29.759649 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73b30a-f0ff-418d-ac95-d6450b941e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:29.759649 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73b30a-f0ff-418d-ac95-d6450b941e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:29.760373 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc73b30a-f0ff-418d-ac95-d6450b941e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:29.762140 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc73b30a-f0ff-418d-ac95-d6450b941e8a admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:29.773718 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 781/1558] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:51:29.777866 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73b30a-f0ff-418d-ac95-d6450b941e8a admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:29.787887 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73b30a-f0ff-418d-ac95-d6450b941e8a admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:29.792438 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72c5ea-706d-4682-8b15-8c920c5e530e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:29.793069 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72c5ea-706d-4682-8b15-8c920c5e530e admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:29.793345 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72c5ea-706d-4682-8b15-8c920c5e530e admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:29.793621 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72c5ea-706d-4682-8b15-8c920c5e530e admin admin] PATH_INFO: `/v3/projects/d3da6026d90b427999eedc39484f26fa/users/eeb8efb3218543e591e11f9530691da7/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:29.808124 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72c5ea-706d-4682-8b15-8c920c5e530e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3da6026d90b427999eedc39484f26fa, user_id=eeb8efb3218543e591e11f9530691da7, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:29.811102 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72c5ea-706d-4682-8b15-8c920c5e530e admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:29.838834 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 778/1559] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:29 2022] PUT /identity/v3/projects/e1bf4b9a5bab4c6ea3c9c24eb93a909f/users/0ea7d079331240e2858d5f546490c6f5/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:51:29.851005 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0157b-f5f8-4f01-b671-f714a2366517 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:29.851263 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0157b-f5f8-4f01-b671-f714a2366517 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:29.851569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0157b-f5f8-4f01-b671-f714a2366517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:29.879458 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-f5d0157b-f5f8-4f01-b671-f714a2366517 None None] MFA Rules not processed for user `0ea7d079331240e2858d5f546490c6f5`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:29.880696 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-f5d0157b-f5f8-4f01-b671-f714a2366517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:29.900892 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:29.923673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 782/1560] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:29 2022] PUT /identity/v3/projects/d3da6026d90b427999eedc39484f26fa/users/eeb8efb3218543e591e11f9530691da7/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:51:29.929302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72c5ea-706d-4682-8b15-8c920c5e530e admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:29.931056 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-bd72c5ea-706d-4682-8b15-8c920c5e530e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:29.948029 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 779/1561] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:51:29.948877 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0157b-f5f8-4f01-b671-f714a2366517 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:30.010170 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72c5ea-706d-4682-8b15-8c920c5e530e admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:30.015711 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ebdb64-b46a-4400-9726-357330c8b431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:30.016443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ebdb64-b46a-4400-9726-357330c8b431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:30.016662 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ebdb64-b46a-4400-9726-357330c8b431 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:30.016973 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ebdb64-b46a-4400-9726-357330c8b431 admin admin] PATH_INFO: `/v3/users/534c283d68754261a521ff28325a5165` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:30.018677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ebdb64-b46a-4400-9726-357330c8b431 admin admin] RBAC: Authorizing `identity:delete_user(user_id=534c283d68754261a521ff28325a5165)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:30.021504 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ebdb64-b46a-4400-9726-357330c8b431 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:30.049153 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0157b-f5f8-4f01-b671-f714a2366517 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:30.057641 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6af22f-75a4-4f39-b9eb-120528bf8fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:30.058658 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6af22f-75a4-4f39-b9eb-120528bf8fc9 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:30.059202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6af22f-75a4-4f39-b9eb-120528bf8fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:30.059678 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6af22f-75a4-4f39-b9eb-120528bf8fc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:30.060366 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6af22f-75a4-4f39-b9eb-120528bf8fc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:30.062492 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6af22f-75a4-4f39-b9eb-120528bf8fc9 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:30.074355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 780/1562] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:30.075093 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6af22f-75a4-4f39-b9eb-120528bf8fc9 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:30.087727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6af22f-75a4-4f39-b9eb-120528bf8fc9 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:30.092774 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac43e5-37d9-48fd-8826-b8508595e71c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:30.093311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac43e5-37d9-48fd-8826-b8508595e71c admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:30.093591 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac43e5-37d9-48fd-8826-b8508595e71c admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:30.093864 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac43e5-37d9-48fd-8826-b8508595e71c admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:30.094646 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac43e5-37d9-48fd-8826-b8508595e71c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:30.096731 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac43e5-37d9-48fd-8826-b8508595e71c admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:30.215421 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 781/1563] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:30 2022] POST /identity/v3/projects => generated 387 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:30.216587 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac43e5-37d9-48fd-8826-b8508595e71c admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:30.234624 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-05ebdb64-b46a-4400-9726-357330c8b431 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '534c283d68754261a521ff28325a5165'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:30.284466 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac43e5-37d9-48fd-8826-b8508595e71c admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:30.289221 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f22ec2-9597-4416-bac6-6595a57e48d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:30.289887 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f22ec2-9597-4416-bac6-6595a57e48d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:30.290175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f22ec2-9597-4416-bac6-6595a57e48d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:30.290444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f22ec2-9597-4416-bac6-6595a57e48d4 admin admin] PATH_INFO: `/v3/projects/d3da6026d90b427999eedc39484f26fa/users/eeb8efb3218543e591e11f9530691da7/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:30.292470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f22ec2-9597-4416-bac6-6595a57e48d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3da6026d90b427999eedc39484f26fa, user_id=eeb8efb3218543e591e11f9530691da7, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:30.295127 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f22ec2-9597-4416-bac6-6595a57e48d4 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:30.307571 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 782/1564] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:30 2022] PUT /identity/v3/projects/d3da6026d90b427999eedc39484f26fa/users/eeb8efb3218543e591e11f9530691da7/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:30.308386 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f22ec2-9597-4416-bac6-6595a57e48d4 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:30.342438 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:30.370809 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f22ec2-9597-4416-bac6-6595a57e48d4 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:30.375265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0987e733-5fae-4b30-a1da-8154c857eff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:30.375893 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987e733-5fae-4b30-a1da-8154c857eff3 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:30.376173 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987e733-5fae-4b30-a1da-8154c857eff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:30.376444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987e733-5fae-4b30-a1da-8154c857eff3 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:30.377067 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0987e733-5fae-4b30-a1da-8154c857eff3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:30.379597 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0987e733-5fae-4b30-a1da-8154c857eff3 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:30.787289 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:30.788723 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-05ebdb64-b46a-4400-9726-357330c8b431 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '534c283d68754261a521ff28325a5165'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:30.808641 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-05ebdb64-b46a-4400-9726-357330c8b431 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '534c283d68754261a521ff28325a5165'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:30.809493 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-05ebdb64-b46a-4400-9726-357330c8b431 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '534c283d68754261a521ff28325a5165'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:30.810162 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-05ebdb64-b46a-4400-9726-357330c8b431 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '534c283d68754261a521ff28325a5165'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:30.823964 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 783/1565] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:51:29 2022] DELETE /identity/v3/users/534c283d68754261a521ff28325a5165 => generated 0 bytes in 895 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:51:30.825665 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621046d2-0a3d-48ef-9f1a-7b288f15ad73 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:30.825665 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621046d2-0a3d-48ef-9f1a-7b288f15ad73 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:30.825827 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621046d2-0a3d-48ef-9f1a-7b288f15ad73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:30.856757 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-621046d2-0a3d-48ef-9f1a-7b288f15ad73 None None] MFA Rules not processed for user `eeb8efb3218543e591e11f9530691da7`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:30.858121 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-621046d2-0a3d-48ef-9f1a-7b288f15ad73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:30.932622 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 784/1566] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:30 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:30.933893 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbc6f7-ebf1-43d1-affe-c8897decf5b4 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:30.934178 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbc6f7-ebf1-43d1-affe-c8897decf5b4 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:30.934452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbc6f7-ebf1-43d1-affe-c8897decf5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:30.959482 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-9dfbc6f7-ebf1-43d1-affe-c8897decf5b4 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:30.960737 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-9dfbc6f7-ebf1-43d1-affe-c8897decf5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:31.017705 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 785/1567] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:31.019554 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfbc6f7-ebf1-43d1-affe-c8897decf5b4 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:31.025748 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-9dfbc6f7-ebf1-43d1-affe-c8897decf5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:31.076176 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfbc6f7-ebf1-43d1-affe-c8897decf5b4 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:31.079781 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0318a3f4-d3c7-413e-b06c-e7bf3e1b8550 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:31.080387 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318a3f4-d3c7-413e-b06c-e7bf3e1b8550 service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:31.080667 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318a3f4-d3c7-413e-b06c-e7bf3e1b8550 service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:31.081193 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0318a3f4-d3c7-413e-b06c-e7bf3e1b8550 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:31.090438 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0318a3f4-d3c7-413e-b06c-e7bf3e1b8550 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:31.091881 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0318a3f4-d3c7-413e-b06c-e7bf3e1b8550 service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:31.101724 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 786/1568] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:31 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:31.102501 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0318a3f4-d3c7-413e-b06c-e7bf3e1b8550 service cinder] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:31.103765 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-0318a3f4-d3c7-413e-b06c-e7bf3e1b8550 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:31.155037 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 783/1569] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:30 2022] POST /identity/v3/users => generated 294 bytes in 845 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:31.156458 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0987e733-5fae-4b30-a1da-8154c857eff3 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:31.159685 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-0987e733-5fae-4b30-a1da-8154c857eff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:31.172643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0318a3f4-d3c7-413e-b06c-e7bf3e1b8550 service cinder] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:31.176587 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0987e733-5fae-4b30-a1da-8154c857eff3 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:31.181805 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b6d171-d5e6-4303-8b69-08b7f65549aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:31.182292 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6ec747-1a1d-4359-a031-7624ee8c00db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:31.182919 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ec747-1a1d-4359-a031-7624ee8c00db service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:31.183043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6d171-d5e6-4303-8b69-08b7f65549aa admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:31.183196 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ec747-1a1d-4359-a031-7624ee8c00db service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:31.183468 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ec747-1a1d-4359-a031-7624ee8c00db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:31.183944 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6d171-d5e6-4303-8b69-08b7f65549aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:31.184433 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6d171-d5e6-4303-8b69-08b7f65549aa admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:31.185184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b6d171-d5e6-4303-8b69-08b7f65549aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:31.187030 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b6d171-d5e6-4303-8b69-08b7f65549aa admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:31.200812 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 784/1570] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:51:31.207125 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6ec747-1a1d-4359-a031-7624ee8c00db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:31.209209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b6d171-d5e6-4303-8b69-08b7f65549aa admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:31.212275 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6ec747-1a1d-4359-a031-7624ee8c00db service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:31.226988 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b6d171-d5e6-4303-8b69-08b7f65549aa admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:31.232273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2addc6ed-9a10-4497-b379-80201b939a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:31.233725 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addc6ed-9a10-4497-b379-80201b939a35 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:31.233725 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addc6ed-9a10-4497-b379-80201b939a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:31.233725 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addc6ed-9a10-4497-b379-80201b939a35 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:31.234218 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2addc6ed-9a10-4497-b379-80201b939a35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:31.236023 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2addc6ed-9a10-4497-b379-80201b939a35 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:31.244427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 787/1571] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:31.250075 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 785/1572] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:31.254144 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6ec747-1a1d-4359-a031-7624ee8c00db service neutron] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:31.279000 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6ec747-1a1d-4359-a031-7624ee8c00db service neutron] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:31.286772 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2988890-4f34-484f-84c9-82ea6d4cd726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:31.287859 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2988890-4f34-484f-84c9-82ea6d4cd726 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:31.288150 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2988890-4f34-484f-84c9-82ea6d4cd726 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:31.288567 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2988890-4f34-484f-84c9-82ea6d4cd726 admin admin] PATH_INFO: `/v3/projects/b9e3d213c9c049bfa91c4fffd5d60f17/users/f0dc9ba6b8cc477badeeb078c883e292/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:31.308452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2988890-4f34-484f-84c9-82ea6d4cd726 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9e3d213c9c049bfa91c4fffd5d60f17, user_id=f0dc9ba6b8cc477badeeb078c883e292, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:31.311484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2988890-4f34-484f-84c9-82ea6d4cd726 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:31.410673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 788/1573] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:31 2022] PUT /identity/v3/projects/b9e3d213c9c049bfa91c4fffd5d60f17/users/f0dc9ba6b8cc477badeeb078c883e292/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:31.421239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7ecd3-94ec-4acd-be74-a529e1b5a6b1 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:31.422029 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7ecd3-94ec-4acd-be74-a529e1b5a6b1 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:31.422586 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a7ecd3-94ec-4acd-be74-a529e1b5a6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:31.454254 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-97a7ecd3-94ec-4acd-be74-a529e1b5a6b1 None None] MFA Rules not processed for user `f0dc9ba6b8cc477badeeb078c883e292`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:31.455464 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-97a7ecd3-94ec-4acd-be74-a529e1b5a6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:31.526671 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 786/1574] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:51:31.528379 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2988890-4f34-484f-84c9-82ea6d4cd726 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:31.612112 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2988890-4f34-484f-84c9-82ea6d4cd726 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:31.618083 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d687b-f5d6-4ee5-8015-0c5807345697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:31.618876 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d687b-f5d6-4ee5-8015-0c5807345697 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:31.619166 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d687b-f5d6-4ee5-8015-0c5807345697 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:31.619571 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976d687b-f5d6-4ee5-8015-0c5807345697 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:31.620265 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d687b-f5d6-4ee5-8015-0c5807345697 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:31.624020 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976d687b-f5d6-4ee5-8015-0c5807345697 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:31.702596 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:31.746502 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 789/1575] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:31 2022] POST /identity/v3/projects => generated 387 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:31.749632 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7ecd3-94ec-4acd-be74-a529e1b5a6b1 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:31.772369 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a7ecd3-94ec-4acd-be74-a529e1b5a6b1 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:31.846102 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d34ad4-f5f7-4d9e-8d90-31a9a4929d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:31.846801 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d34ad4-f5f7-4d9e-8d90-31a9a4929d31 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:31.847833 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d34ad4-f5f7-4d9e-8d90-31a9a4929d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:31.847833 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d34ad4-f5f7-4d9e-8d90-31a9a4929d31 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:31.848294 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d34ad4-f5f7-4d9e-8d90-31a9a4929d31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:31.850742 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d34ad4-f5f7-4d9e-8d90-31a9a4929d31 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:31.977593 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 787/1576] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:31 2022] POST /identity/v3/users => generated 300 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:31.981311 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d687b-f5d6-4ee5-8015-0c5807345697 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:32.010158 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976d687b-f5d6-4ee5-8015-0c5807345697 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:32.020301 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c100e9d-9a97-4d7c-b301-d14d5561b1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:32.020878 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c100e9d-9a97-4d7c-b301-d14d5561b1f5 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:32.021157 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c100e9d-9a97-4d7c-b301-d14d5561b1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:32.021349 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c100e9d-9a97-4d7c-b301-d14d5561b1f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:32.022006 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c100e9d-9a97-4d7c-b301-d14d5561b1f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:32.023901 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c100e9d-9a97-4d7c-b301-d14d5561b1f5 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:32.038184 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 790/1577] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:32.039350 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d34ad4-f5f7-4d9e-8d90-31a9a4929d31 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:32.049638 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d34ad4-f5f7-4d9e-8d90-31a9a4929d31 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:32.055262 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c710a4-b51e-4777-8911-13912dc98723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:32.056059 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c710a4-b51e-4777-8911-13912dc98723 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:32.056498 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c710a4-b51e-4777-8911-13912dc98723 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:32.056738 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c710a4-b51e-4777-8911-13912dc98723 admin admin] PATH_INFO: `/v3/projects/e04fa65e110a435892939347d958e58f/users/a66ee61c4d5b47209a4f1eb90809f38e/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:32.073550 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c710a4-b51e-4777-8911-13912dc98723 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e04fa65e110a435892939347d958e58f, user_id=a66ee61c4d5b47209a4f1eb90809f38e, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:32.076389 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c710a4-b51e-4777-8911-13912dc98723 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:32.078541 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:32.127845 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c100e9d-9a97-4d7c-b301-d14d5561b1f5 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:32.127845 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-0c100e9d-9a97-4d7c-b301-d14d5561b1f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:32.136077 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c100e9d-9a97-4d7c-b301-d14d5561b1f5 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:32.140822 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbc0e8-24f0-4b68-897f-9d611c310978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:32.141657 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbc0e8-24f0-4b68-897f-9d611c310978 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:32.141939 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbc0e8-24f0-4b68-897f-9d611c310978 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:32.142208 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbc0e8-24f0-4b68-897f-9d611c310978 admin admin] PATH_INFO: `/v3/projects/1644e7940fa1423f89ad3654b27e905e` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:32.143574 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cbc0e8-24f0-4b68-897f-9d611c310978 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1644e7940fa1423f89ad3654b27e905e)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:32.145634 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cbc0e8-24f0-4b68-897f-9d611c310978 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:32.203346 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 788/1578] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:32 2022] PUT /identity/v3/projects/e04fa65e110a435892939347d958e58f/users/a66ee61c4d5b47209a4f1eb90809f38e/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:32.210766 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c710a4-b51e-4777-8911-13912dc98723 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:32.274541 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c710a4-b51e-4777-8911-13912dc98723 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:32.279353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c2558c-f56a-4591-9eed-93a10cb8ef5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:32.280044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2558c-f56a-4591-9eed-93a10cb8ef5d admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:32.280306 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2558c-f56a-4591-9eed-93a10cb8ef5d admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:32.280584 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2558c-f56a-4591-9eed-93a10cb8ef5d admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:32.281257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c2558c-f56a-4591-9eed-93a10cb8ef5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:32.282856 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c2558c-f56a-4591-9eed-93a10cb8ef5d admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:32.293674 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 789/1579] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:51:32.294592 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c2558c-f56a-4591-9eed-93a10cb8ef5d admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:32.302197 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-23cbc0e8-24f0-4b68-897f-9d611c310978 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1644e7940fa1423f89ad3654b27e905e', 'request_id': 'req-23cbc0e8-24f0-4b68-897f-9d611c310978'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:32.356175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c2558c-f56a-4591-9eed-93a10cb8ef5d admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:32.366863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-23cbc0e8-24f0-4b68-897f-9d611c310978 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1644e7940fa1423f89ad3654b27e905e', 'request_id': 'req-23cbc0e8-24f0-4b68-897f-9d611c310978'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:32.373091 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-23cbc0e8-24f0-4b68-897f-9d611c310978 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1644e7940fa1423f89ad3654b27e905e', 'request_id': 'req-23cbc0e8-24f0-4b68-897f-9d611c310978'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:32.384584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-23cbc0e8-24f0-4b68-897f-9d611c310978 admin admin] The token cache is being invalidate because project 1644e7940fa1423f89ad3654b27e905e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:32.384880 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-23cbc0e8-24f0-4b68-897f-9d611c310978 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:32.388061 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-23cbc0e8-24f0-4b68-897f-9d611c310978 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:32.406359 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 791/1580] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:32 2022] DELETE /identity/v3/projects/1644e7940fa1423f89ad3654b27e905e => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:51:32.410007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbc0e8-24f0-4b68-897f-9d611c310978 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:32.417729 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-23cbc0e8-24f0-4b68-897f-9d611c310978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:32.417884 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066d5b1-0607-4750-b9f7-6bc3ac3043e1 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeb8efb3218543e591e11f9530691da7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3da6026d90b427999eedc39484f26fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:32.417884 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066d5b1-0607-4750-b9f7-6bc3ac3043e1 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:32.417884 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066d5b1-0607-4750-b9f7-6bc3ac3043e1 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:32.418101 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066d5b1-0607-4750-b9f7-6bc3ac3043e1 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:32.418101 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066d5b1-0607-4750-b9f7-6bc3ac3043e1 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:51:32.418253 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066d5b1-0607-4750-b9f7-6bc3ac3043e1 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:32.420186 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3066d5b1-0607-4750-b9f7-6bc3ac3043e1 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:32.439087 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 790/1581] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:51:32.439704 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066d5b1-0607-4750-b9f7-6bc3ac3043e1 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:32.451722 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-3066d5b1-0607-4750-b9f7-6bc3ac3043e1 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:32.502678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbc0e8-24f0-4b68-897f-9d611c310978 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:32.507975 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18bd69-5f18-4184-97a0-1c15ad2bcf04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:32.508728 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18bd69-5f18-4184-97a0-1c15ad2bcf04 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:32.509075 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18bd69-5f18-4184-97a0-1c15ad2bcf04 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:32.510210 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c18bd69-5f18-4184-97a0-1c15ad2bcf04 admin admin] PATH_INFO: `/v3/projects/e04fa65e110a435892939347d958e58f/users/a66ee61c4d5b47209a4f1eb90809f38e/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:32.511651 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c18bd69-5f18-4184-97a0-1c15ad2bcf04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e04fa65e110a435892939347d958e58f, user_id=a66ee61c4d5b47209a4f1eb90809f38e, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:32.514418 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c18bd69-5f18-4184-97a0-1c15ad2bcf04 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:32.523444 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:32.548217 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066d5b1-0607-4750-b9f7-6bc3ac3043e1 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:32.559858 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 792/1582] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:32 2022] PUT /identity/v3/projects/e04fa65e110a435892939347d958e58f/users/a66ee61c4d5b47209a4f1eb90809f38e/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:32.561881 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18bd69-5f18-4184-97a0-1c15ad2bcf04 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:32.565383 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-2c18bd69-5f18-4184-97a0-1c15ad2bcf04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:32.633993 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a882d-48da-4a86-9b37-083513b8de18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:32.635183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a882d-48da-4a86-9b37-083513b8de18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:32.635473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a882d-48da-4a86-9b37-083513b8de18 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:32.635769 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a882d-48da-4a86-9b37-083513b8de18 admin admin] PATH_INFO: `/v3/users/b10b44d2f74043c8bdd5b96912ccd54d` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:32.637216 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a882d-48da-4a86-9b37-083513b8de18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b10b44d2f74043c8bdd5b96912ccd54d)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:32.639534 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a882d-48da-4a86-9b37-083513b8de18 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:32.704958 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c18bd69-5f18-4184-97a0-1c15ad2bcf04 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:32.710608 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f880d-5471-41a5-a9a1-4247cb5c7991 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeb8efb3218543e591e11f9530691da7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3da6026d90b427999eedc39484f26fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:32.711230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f880d-5471-41a5-a9a1-4247cb5c7991 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:32.711519 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f880d-5471-41a5-a9a1-4247cb5c7991 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:32.711827 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f880d-5471-41a5-a9a1-4247cb5c7991 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:32.712473 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f880d-5471-41a5-a9a1-4247cb5c7991 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:32.714440 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f880d-5471-41a5-a9a1-4247cb5c7991 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:32.738714 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:32.796857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 793/1583] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:32 2022] POST /identity/v3/projects => generated 355 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:32.798212 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b31a3-c87b-415b-9a25-5b0bb14963e5 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:32.798508 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b31a3-c87b-415b-9a25-5b0bb14963e5 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:32.799639 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b31a3-c87b-415b-9a25-5b0bb14963e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:32.826370 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-a03b31a3-c87b-415b-9a25-5b0bb14963e5 None None] MFA Rules not processed for user `a66ee61c4d5b47209a4f1eb90809f38e`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:32.827585 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-a03b31a3-c87b-415b-9a25-5b0bb14963e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:32.888374 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-953a882d-48da-4a86-9b37-083513b8de18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b10b44d2f74043c8bdd5b96912ccd54d'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:32.889465 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-953a882d-48da-4a86-9b37-083513b8de18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b10b44d2f74043c8bdd5b96912ccd54d'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:32.901497 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 794/1584] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:32 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:32.903019 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b31a3-c87b-415b-9a25-5b0bb14963e5 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:32.966129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-953a882d-48da-4a86-9b37-083513b8de18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b10b44d2f74043c8bdd5b96912ccd54d'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:32.989914 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-953a882d-48da-4a86-9b37-083513b8de18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b10b44d2f74043c8bdd5b96912ccd54d'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:32.991440 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-953a882d-48da-4a86-9b37-083513b8de18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b10b44d2f74043c8bdd5b96912ccd54d'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:33.013512 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b31a3-c87b-415b-9a25-5b0bb14963e5 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:33.020427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f555e6-5c30-4fc8-883e-78938438a732 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeb8efb3218543e591e11f9530691da7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3da6026d90b427999eedc39484f26fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:33.021072 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f555e6-5c30-4fc8-883e-78938438a732 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:33.021465 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f555e6-5c30-4fc8-883e-78938438a732 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:33.021751 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f555e6-5c30-4fc8-883e-78938438a732 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:33.025339 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f555e6-5c30-4fc8-883e-78938438a732 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:51:33.025494 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 791/1585] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:51:32 2022] DELETE /identity/v3/users/b10b44d2f74043c8bdd5b96912ccd54d => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:33.025589 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f555e6-5c30-4fc8-883e-78938438a732 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:33.025673 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a882d-48da-4a86-9b37-083513b8de18 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:33.025754 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f555e6-5c30-4fc8-883e-78938438a732 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:33.025952 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-953a882d-48da-4a86-9b37-083513b8de18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:33.043550 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 795/1586] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:51:33.137889 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a882d-48da-4a86-9b37-083513b8de18 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:33.141713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7abab8-b12f-4e58-a297-be86cdc54f5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:33.142337 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7abab8-b12f-4e58-a297-be86cdc54f5d service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:33.144704 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7abab8-b12f-4e58-a297-be86cdc54f5d service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:33.145604 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7abab8-b12f-4e58-a297-be86cdc54f5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:33.150867 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-8b7abab8-b12f-4e58-a297-be86cdc54f5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:33.247900 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7abab8-b12f-4e58-a297-be86cdc54f5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:33.250153 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7abab8-b12f-4e58-a297-be86cdc54f5d service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:33.272081 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 792/1587] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:33 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:33.274983 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f555e6-5c30-4fc8-883e-78938438a732 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:33.278137 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-72f555e6-5c30-4fc8-883e-78938438a732 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:33.305350 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f555e6-5c30-4fc8-883e-78938438a732 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:33.386261 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29020d3-dcce-4e89-a035-d7560a2c7422 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eeb8efb3218543e591e11f9530691da7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd3da6026d90b427999eedc39484f26fa', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:33.389004 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29020d3-dcce-4e89-a035-d7560a2c7422 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:33.389298 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29020d3-dcce-4e89-a035-d7560a2c7422 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:33.389602 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29020d3-dcce-4e89-a035-d7560a2c7422 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] PATH_INFO: `/v3/projects/d575cf9e02c349918f572501b5af9914` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:33.390975 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29020d3-dcce-4e89-a035-d7560a2c7422 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d575cf9e02c349918f572501b5af9914)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:33.393481 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29020d3-dcce-4e89-a035-d7560a2c7422 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:33.521267 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c29020d3-dcce-4e89-a035-d7560a2c7422 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd575cf9e02c349918f572501b5af9914', 'request_id': 'req-c29020d3-dcce-4e89-a035-d7560a2c7422'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:33.523339 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:33.639388 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:33.641867 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c29020d3-dcce-4e89-a035-d7560a2c7422 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd575cf9e02c349918f572501b5af9914', 'request_id': 'req-c29020d3-dcce-4e89-a035-d7560a2c7422'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:33.652921 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c29020d3-dcce-4e89-a035-d7560a2c7422 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd575cf9e02c349918f572501b5af9914', 'request_id': 'req-c29020d3-dcce-4e89-a035-d7560a2c7422'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:33.669845 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c29020d3-dcce-4e89-a035-d7560a2c7422 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] The token cache is being invalidate because project d575cf9e02c349918f572501b5af9914 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:33.670538 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c29020d3-dcce-4e89-a035-d7560a2c7422 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:33.671431 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c29020d3-dcce-4e89-a035-d7560a2c7422 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:33.678238 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 796/1588] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:33 2022] DELETE /identity/v3/projects/d575cf9e02c349918f572501b5af9914 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:33.964433 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7abab8-b12f-4e58-a297-be86cdc54f5d service cinder] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:33.966410 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-8b7abab8-b12f-4e58-a297-be86cdc54f5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:34.039687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7abab8-b12f-4e58-a297-be86cdc54f5d service cinder] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:34.099976 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5be77c-d7c4-4060-8b64-d05b1f3f2b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:34.100622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5be77c-d7c4-4060-8b64-d05b1f3f2b43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:34.100902 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5be77c-d7c4-4060-8b64-d05b1f3f2b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:34.101176 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5be77c-d7c4-4060-8b64-d05b1f3f2b43 admin admin] PATH_INFO: `/v3/projects/9df46be471214add89a6bb3651530cc5` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:34.102312 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5be77c-d7c4-4060-8b64-d05b1f3f2b43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9df46be471214add89a6bb3651530cc5)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:34.104255 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5be77c-d7c4-4060-8b64-d05b1f3f2b43 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:34.301033 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3a5be77c-d7c4-4060-8b64-d05b1f3f2b43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9df46be471214add89a6bb3651530cc5', 'request_id': 'req-3a5be77c-d7c4-4060-8b64-d05b1f3f2b43'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:34.348742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29020d3-dcce-4e89-a035-d7560a2c7422 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:34.350228 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c29020d3-dcce-4e89-a035-d7560a2c7422 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:34.363325 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29020d3-dcce-4e89-a035-d7560a2c7422 tempest-VolumeQuotaClassesTest-2057031313 tempest-VolumeQuotaClassesTest-2057031313-project-admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:34.397896 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:34.398495 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3a5be77c-d7c4-4060-8b64-d05b1f3f2b43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9df46be471214add89a6bb3651530cc5', 'request_id': 'req-3a5be77c-d7c4-4060-8b64-d05b1f3f2b43'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:34.407039 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3a5be77c-d7c4-4060-8b64-d05b1f3f2b43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9df46be471214add89a6bb3651530cc5', 'request_id': 'req-3a5be77c-d7c4-4060-8b64-d05b1f3f2b43'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:34.421481 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3a5be77c-d7c4-4060-8b64-d05b1f3f2b43 admin admin] The token cache is being invalidate because project 9df46be471214add89a6bb3651530cc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:34.422294 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3a5be77c-d7c4-4060-8b64-d05b1f3f2b43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:34.423018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3a5be77c-d7c4-4060-8b64-d05b1f3f2b43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:34.424652 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 793/1589] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:33 2022] DELETE /identity/v3/projects/9df46be471214add89a6bb3651530cc5 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:34.429039 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb969f52-bd50-46de-abef-1b11b81ba09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:34.429598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77380f98-39a0-40c2-90eb-fcba40a03bfd None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:34.429982 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77380f98-39a0-40c2-90eb-fcba40a03bfd None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:34.430377 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77380f98-39a0-40c2-90eb-fcba40a03bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:34.431022 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb969f52-bd50-46de-abef-1b11b81ba09b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:34.431315 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb969f52-bd50-46de-abef-1b11b81ba09b admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:34.431725 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb969f52-bd50-46de-abef-1b11b81ba09b admin admin] PATH_INFO: `/v3/users/e292f77e8d704772b9becd5171fbec21` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:34.432959 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb969f52-bd50-46de-abef-1b11b81ba09b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e292f77e8d704772b9becd5171fbec21)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:34.434910 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb969f52-bd50-46de-abef-1b11b81ba09b admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:34.457048 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-77380f98-39a0-40c2-90eb-fcba40a03bfd None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:34.458398 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-77380f98-39a0-40c2-90eb-fcba40a03bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:34.478254 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 794/1590] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:34.486074 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ca197-8674-4ad4-96e2-409a111880be None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:34.486353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ca197-8674-4ad4-96e2-409a111880be None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:34.486856 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ca197-8674-4ad4-96e2-409a111880be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:34.513608 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-795ca197-8674-4ad4-96e2-409a111880be None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:34.515267 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-795ca197-8674-4ad4-96e2-409a111880be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:34.525628 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 795/1591] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:34.534362 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ca197-8674-4ad4-96e2-409a111880be None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:34.545094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ca197-8674-4ad4-96e2-409a111880be None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:34.552242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df0110a-7892-49ae-86a7-3df92631985b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:34.553192 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0110a-7892-49ae-86a7-3df92631985b admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:34.553192 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0110a-7892-49ae-86a7-3df92631985b admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:34.553415 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df0110a-7892-49ae-86a7-3df92631985b admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:34.554059 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df0110a-7892-49ae-86a7-3df92631985b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:51:34.554360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df0110a-7892-49ae-86a7-3df92631985b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:34.556172 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df0110a-7892-49ae-86a7-3df92631985b admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:34.571821 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 796/1592] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:51:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:51:34.576033 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df0110a-7892-49ae-86a7-3df92631985b admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:34.589261 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df0110a-7892-49ae-86a7-3df92631985b admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:34.598902 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe63990-0266-4d7e-99be-ab846cdfea87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:34.599768 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe63990-0266-4d7e-99be-ab846cdfea87 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:34.600971 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe63990-0266-4d7e-99be-ab846cdfea87 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:34.602051 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe63990-0266-4d7e-99be-ab846cdfea87 admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:34.603206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe63990-0266-4d7e-99be-ab846cdfea87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:34.608794 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe63990-0266-4d7e-99be-ab846cdfea87 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:34.684027 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-bb969f52-bd50-46de-abef-1b11b81ba09b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e292f77e8d704772b9becd5171fbec21'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:34.792015 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-bb969f52-bd50-46de-abef-1b11b81ba09b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e292f77e8d704772b9becd5171fbec21'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:34.793023 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:34.812598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 797/1593] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:34 2022] POST /identity/v3/projects => generated 385 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:51:34.819914 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe63990-0266-4d7e-99be-ab846cdfea87 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:34.835780 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-bb969f52-bd50-46de-abef-1b11b81ba09b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e292f77e8d704772b9becd5171fbec21'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:34.836974 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-bb969f52-bd50-46de-abef-1b11b81ba09b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e292f77e8d704772b9becd5171fbec21'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:34.838980 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-bb969f52-bd50-46de-abef-1b11b81ba09b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e292f77e8d704772b9becd5171fbec21'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:34.869906 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 797/1594] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:51:34 2022] DELETE /identity/v3/users/e292f77e8d704772b9becd5171fbec21 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:51:34.874834 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dddea-da3b-4b75-bfe7-b90b75ff02ce None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:34.877756 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dddea-da3b-4b75-bfe7-b90b75ff02ce None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:34.879414 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dddea-da3b-4b75-bfe7-b90b75ff02ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:34.910836 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-799dddea-da3b-4b75-bfe7-b90b75ff02ce None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:34.913181 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-799dddea-da3b-4b75-bfe7-b90b75ff02ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:34.991434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe63990-0266-4d7e-99be-ab846cdfea87 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:35.037104 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 798/1595] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:35.046241 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799dddea-da3b-4b75-bfe7-b90b75ff02ce None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:35.048061 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-799dddea-da3b-4b75-bfe7-b90b75ff02ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:35.091184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d94fc-5f81-4862-ab1c-9df0ea4f60ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:35.092665 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d94fc-5f81-4862-ab1c-9df0ea4f60ac admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:35.092665 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d94fc-5f81-4862-ab1c-9df0ea4f60ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:35.092665 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0d94fc-5f81-4862-ab1c-9df0ea4f60ac admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:35.093593 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d94fc-5f81-4862-ab1c-9df0ea4f60ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:35.096197 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0d94fc-5f81-4862-ab1c-9df0ea4f60ac admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:35.162007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799dddea-da3b-4b75-bfe7-b90b75ff02ce None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:35.165871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d7c37-8676-4bf9-ba64-0c9d99e398e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:35.166503 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d7c37-8676-4bf9-ba64-0c9d99e398e7 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:35.166827 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d7c37-8676-4bf9-ba64-0c9d99e398e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:35.167311 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d7c37-8676-4bf9-ba64-0c9d99e398e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:35.184487 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d7c37-8676-4bf9-ba64-0c9d99e398e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:35.185922 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d7c37-8676-4bf9-ba64-0c9d99e398e7 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:35.213772 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 799/1596] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:35.214720 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 798/1597] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:34 2022] POST /identity/v3/users => generated 293 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:35.218111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d7c37-8676-4bf9-ba64-0c9d99e398e7 service neutron] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:35.219121 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d94fc-5f81-4862-ab1c-9df0ea4f60ac admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:35.219661 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-b21d7c37-8676-4bf9-ba64-0c9d99e398e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:35.221465 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-8f0d94fc-5f81-4862-ab1c-9df0ea4f60ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:35.236499 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d7c37-8676-4bf9-ba64-0c9d99e398e7 service neutron] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:35.238359 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0d94fc-5f81-4862-ab1c-9df0ea4f60ac admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:35.243334 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893414b-a645-42a5-9412-ef211b0501f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:35.244152 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893414b-a645-42a5-9412-ef211b0501f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:35.244335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893414b-a645-42a5-9412-ef211b0501f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:35.244575 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893414b-a645-42a5-9412-ef211b0501f7 admin admin] PATH_INFO: `/v3/users/f877bfbb85b24b5a8bc98516066f4164` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:35.245867 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893414b-a645-42a5-9412-ef211b0501f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f877bfbb85b24b5a8bc98516066f4164)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:35.247018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b83c3a-51d4-453f-9e14-465a10a96ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:35.247917 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b83c3a-51d4-453f-9e14-465a10a96ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:35.248135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893414b-a645-42a5-9412-ef211b0501f7 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:35.248525 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b83c3a-51d4-453f-9e14-465a10a96ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:35.248959 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b83c3a-51d4-453f-9e14-465a10a96ab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:35.249766 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b83c3a-51d4-453f-9e14-465a10a96ab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:35.253609 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b83c3a-51d4-453f-9e14-465a10a96ab3 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:35.292110 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 799/1598] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:51:35.293713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b83c3a-51d4-453f-9e14-465a10a96ab3 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:35.304956 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b83c3a-51d4-453f-9e14-465a10a96ab3 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:35.309725 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14139696-46f6-4d51-97db-e98fd1056512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:35.310344 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14139696-46f6-4d51-97db-e98fd1056512 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:35.310628 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14139696-46f6-4d51-97db-e98fd1056512 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:35.310902 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14139696-46f6-4d51-97db-e98fd1056512 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:35.311633 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14139696-46f6-4d51-97db-e98fd1056512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:35.313337 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14139696-46f6-4d51-97db-e98fd1056512 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:35.325561 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 800/1599] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:35.330381 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14139696-46f6-4d51-97db-e98fd1056512 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:35.341113 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14139696-46f6-4d51-97db-e98fd1056512 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:35.346476 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d849177a-fc5d-423b-a69b-c75a66569521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:35.349142 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849177a-fc5d-423b-a69b-c75a66569521 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:35.349142 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849177a-fc5d-423b-a69b-c75a66569521 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:35.349142 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d849177a-fc5d-423b-a69b-c75a66569521 admin admin] PATH_INFO: `/v3/projects/c0e74d2edc8540868fd10f0efd97b9e9/users/32b8473a2d0540ff842f0a9fede4a38e/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:35.363588 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d849177a-fc5d-423b-a69b-c75a66569521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0e74d2edc8540868fd10f0efd97b9e9, user_id=32b8473a2d0540ff842f0a9fede4a38e, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:35.366302 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d849177a-fc5d-423b-a69b-c75a66569521 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:35.462521 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 801/1600] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:35 2022] PUT /identity/v3/projects/c0e74d2edc8540868fd10f0efd97b9e9/users/32b8473a2d0540ff842f0a9fede4a38e/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:35.472688 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693f174-11d1-46f5-8d15-3fc05fede46a None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:35.472688 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693f174-11d1-46f5-8d15-3fc05fede46a None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:35.472688 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693f174-11d1-46f5-8d15-3fc05fede46a None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:35.472688 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 802/1601] 10.222.0.17 () {54 vars in 921 bytes} [Mon Oct 17 03:51:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:51:35.475402 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02507d82-cd10-408d-8320-b7a0e7ce284a None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:35.475934 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02507d82-cd10-408d-8320-b7a0e7ce284a None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:35.476349 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02507d82-cd10-408d-8320-b7a0e7ce284a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:35.545050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-02507d82-cd10-408d-8320-b7a0e7ce284a None None] MFA Rules not processed for user `32b8473a2d0540ff842f0a9fede4a38e`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:35.546234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-02507d82-cd10-408d-8320-b7a0e7ce284a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:35.560156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-e893414b-a645-42a5-9412-ef211b0501f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f877bfbb85b24b5a8bc98516066f4164'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:35.618701 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 803/1602] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:51:35.620349 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a87189-f48c-4d37-ad6a-618b45836fc0 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:35.620709 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a87189-f48c-4d37-ad6a-618b45836fc0 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:35.621063 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a87189-f48c-4d37-ad6a-618b45836fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:35.640996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-21a87189-f48c-4d37-ad6a-618b45836fc0 None None] MFA Rules not processed for user `d961b751763a4e49b5d4ca4c795c5bf6`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:35.645241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-21a87189-f48c-4d37-ad6a-618b45836fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:35.663831 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-e893414b-a645-42a5-9412-ef211b0501f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f877bfbb85b24b5a8bc98516066f4164'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:35.690957 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-e893414b-a645-42a5-9412-ef211b0501f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f877bfbb85b24b5a8bc98516066f4164'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:35.691874 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-e893414b-a645-42a5-9412-ef211b0501f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f877bfbb85b24b5a8bc98516066f4164'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:35.692631 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-e893414b-a645-42a5-9412-ef211b0501f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f877bfbb85b24b5a8bc98516066f4164'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:35.714653 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 800/1603] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:51:35 2022] DELETE /identity/v3/users/f877bfbb85b24b5a8bc98516066f4164 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:35.714840 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893414b-a645-42a5-9412-ef211b0501f7 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:35.716722 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-e893414b-a645-42a5-9412-ef211b0501f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:35.719171 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 804/1604] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Oct 17 03:51:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:35.720057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a87189-f48c-4d37-ad6a-618b45836fc0 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:35.721422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-21a87189-f48c-4d37-ad6a-618b45836fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:35.801440 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893414b-a645-42a5-9412-ef211b0501f7 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:35.803577 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a87189-f48c-4d37-ad6a-618b45836fc0 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:35.806602 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d44c08-3847-4d0a-833b-6fa53d48606e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:35.810584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d44c08-3847-4d0a-833b-6fa53d48606e service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:35.810584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d44c08-3847-4d0a-833b-6fa53d48606e service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:35.810584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d44c08-3847-4d0a-833b-6fa53d48606e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:35.810868 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d38878-60f2-478e-83c2-b1785763b855 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:35.811029 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-d9d44c08-3847-4d0a-833b-6fa53d48606e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:35.811141 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d38878-60f2-478e-83c2-b1785763b855 service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:35.811141 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d38878-60f2-478e-83c2-b1785763b855 service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:35.811471 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d38878-60f2-478e-83c2-b1785763b855 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:35.813560 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-40d38878-60f2-478e-83c2-b1785763b855 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:35.895513 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d38878-60f2-478e-83c2-b1785763b855 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:35.897155 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d38878-60f2-478e-83c2-b1785763b855 service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:35.904125 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d44c08-3847-4d0a-833b-6fa53d48606e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:35.905888 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d44c08-3847-4d0a-833b-6fa53d48606e service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:35.908063 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 805/1605] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:35.910247 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3165a05-b3ad-4912-bd7a-e99d5d4e3031 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:35.910671 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3165a05-b3ad-4912-bd7a-e99d5d4e3031 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:35.911029 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3165a05-b3ad-4912-bd7a-e99d5d4e3031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:35.918325 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 801/1606] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:35.919189 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d44c08-3847-4d0a-833b-6fa53d48606e service cinder] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:35.921157 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-d9d44c08-3847-4d0a-833b-6fa53d48606e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:35.974042 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-e3165a05-b3ad-4912-bd7a-e99d5d4e3031 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:35.975525 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-e3165a05-b3ad-4912-bd7a-e99d5d4e3031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:36.048916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d44c08-3847-4d0a-833b-6fa53d48606e service cinder] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:36.056905 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d065654-c4ff-4a1d-8cf4-7365d9ace8b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:36.058357 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d065654-c4ff-4a1d-8cf4-7365d9ace8b1 service nova] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:36.058653 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d065654-c4ff-4a1d-8cf4-7365d9ace8b1 service nova] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:36.058930 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d065654-c4ff-4a1d-8cf4-7365d9ace8b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:36.072432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d065654-c4ff-4a1d-8cf4-7365d9ace8b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:36.074240 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d065654-c4ff-4a1d-8cf4-7365d9ace8b1 service nova] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:36.098118 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 802/1607] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:36.101374 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:36.114209 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d065654-c4ff-4a1d-8cf4-7365d9ace8b1 service nova] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:36.117034 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-6d065654-c4ff-4a1d-8cf4-7365d9ace8b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:36.136012 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 806/1608] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:36.145659 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3165a05-b3ad-4912-bd7a-e99d5d4e3031 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:36.148441 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-e3165a05-b3ad-4912-bd7a-e99d5d4e3031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:36.226475 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3165a05-b3ad-4912-bd7a-e99d5d4e3031 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:36.230108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d065654-c4ff-4a1d-8cf4-7365d9ace8b1 service nova] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:36.235200 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b79af-0f34-4aac-b23f-49cf9aa9b536 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:36.235200 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b79af-0f34-4aac-b23f-49cf9aa9b536 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:36.235200 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b79af-0f34-4aac-b23f-49cf9aa9b536 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:36.235200 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b79af-0f34-4aac-b23f-49cf9aa9b536 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:36.235884 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d773fb-8850-4613-98c2-bc2825a74b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:36.236673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d773fb-8850-4613-98c2-bc2825a74b9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:36.236955 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d773fb-8850-4613-98c2-bc2825a74b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:36.237228 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d773fb-8850-4613-98c2-bc2825a74b9d admin admin] PATH_INFO: `/v3/projects/e7cab2a3bf1849e9845c27da6f0c4c90` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:36.238558 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d773fb-8850-4613-98c2-bc2825a74b9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7cab2a3bf1849e9845c27da6f0c4c90)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:36.241466 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d773fb-8850-4613-98c2-bc2825a74b9d admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:36.244439 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b79af-0f34-4aac-b23f-49cf9aa9b536 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:36.246487 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b79af-0f34-4aac-b23f-49cf9aa9b536 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:36.271185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 807/1609] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:36.390409 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-98d773fb-8850-4613-98c2-bc2825a74b9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7cab2a3bf1849e9845c27da6f0c4c90', 'request_id': 'req-98d773fb-8850-4613-98c2-bc2825a74b9d'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:36.539253 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-98d773fb-8850-4613-98c2-bc2825a74b9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7cab2a3bf1849e9845c27da6f0c4c90', 'request_id': 'req-98d773fb-8850-4613-98c2-bc2825a74b9d'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:36.547238 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-98d773fb-8850-4613-98c2-bc2825a74b9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7cab2a3bf1849e9845c27da6f0c4c90', 'request_id': 'req-98d773fb-8850-4613-98c2-bc2825a74b9d'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:36.560032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-98d773fb-8850-4613-98c2-bc2825a74b9d admin admin] The token cache is being invalidate because project e7cab2a3bf1849e9845c27da6f0c4c90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:36.560263 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-98d773fb-8850-4613-98c2-bc2825a74b9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:36.560986 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-98d773fb-8850-4613-98c2-bc2825a74b9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:36.562685 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 803/1610] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:36 2022] DELETE /identity/v3/projects/e7cab2a3bf1849e9845c27da6f0c4c90 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:36.567306 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b79af-0f34-4aac-b23f-49cf9aa9b536 service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:36.577061 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b79af-0f34-4aac-b23f-49cf9aa9b536 service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:36.634283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab910a79-c2e4-4506-b7f2-cf37236746b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:36.635013 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab910a79-c2e4-4506-b7f2-cf37236746b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:36.635400 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab910a79-c2e4-4506-b7f2-cf37236746b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:36.637423 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab910a79-c2e4-4506-b7f2-cf37236746b9 admin admin] PATH_INFO: `/v3/users/eeb8efb3218543e591e11f9530691da7` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:36.637423 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab910a79-c2e4-4506-b7f2-cf37236746b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eeb8efb3218543e591e11f9530691da7)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:36.639058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab910a79-c2e4-4506-b7f2-cf37236746b9 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:36.832843 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:36.846271 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ab910a79-c2e4-4506-b7f2-cf37236746b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eeb8efb3218543e591e11f9530691da7'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:36.847936 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ab910a79-c2e4-4506-b7f2-cf37236746b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eeb8efb3218543e591e11f9530691da7'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:36.925426 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ab910a79-c2e4-4506-b7f2-cf37236746b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eeb8efb3218543e591e11f9530691da7'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:36.928769 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d773fb-8850-4613-98c2-bc2825a74b9d admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:36.930157 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-98d773fb-8850-4613-98c2-bc2825a74b9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:36.944545 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d773fb-8850-4613-98c2-bc2825a74b9d admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:36.950015 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67424e4a-b730-4912-9151-3974eb9591a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:36.950982 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67424e4a-b730-4912-9151-3974eb9591a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:36.951264 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67424e4a-b730-4912-9151-3974eb9591a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:36.951542 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67424e4a-b730-4912-9151-3974eb9591a2 admin admin] PATH_INFO: `/v3/projects/785f10a2c39946ae8d812e4857a1e8d1` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:36.953173 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67424e4a-b730-4912-9151-3974eb9591a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=785f10a2c39946ae8d812e4857a1e8d1)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:36.954750 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67424e4a-b730-4912-9151-3974eb9591a2 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:36.955203 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ab910a79-c2e4-4506-b7f2-cf37236746b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eeb8efb3218543e591e11f9530691da7'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:36.956431 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ab910a79-c2e4-4506-b7f2-cf37236746b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eeb8efb3218543e591e11f9530691da7'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:36.991606 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 808/1611] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:51:36 2022] DELETE /identity/v3/users/eeb8efb3218543e591e11f9530691da7 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:37.116450 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-67424e4a-b730-4912-9151-3974eb9591a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '785f10a2c39946ae8d812e4857a1e8d1', 'request_id': 'req-67424e4a-b730-4912-9151-3974eb9591a2'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:37.230705 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-67424e4a-b730-4912-9151-3974eb9591a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '785f10a2c39946ae8d812e4857a1e8d1', 'request_id': 'req-67424e4a-b730-4912-9151-3974eb9591a2'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:37.243962 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-67424e4a-b730-4912-9151-3974eb9591a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '785f10a2c39946ae8d812e4857a1e8d1', 'request_id': 'req-67424e4a-b730-4912-9151-3974eb9591a2'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:37.258930 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-67424e4a-b730-4912-9151-3974eb9591a2 admin admin] The token cache is being invalidate because project 785f10a2c39946ae8d812e4857a1e8d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:37.259304 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-67424e4a-b730-4912-9151-3974eb9591a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:37.260050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-67424e4a-b730-4912-9151-3974eb9591a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:37.261841 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 804/1612] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:36 2022] DELETE /identity/v3/projects/785f10a2c39946ae8d812e4857a1e8d1 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:51:37.265725 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab910a79-c2e4-4506-b7f2-cf37236746b9 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:37.267305 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-ab910a79-c2e4-4506-b7f2-cf37236746b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:37.366626 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab910a79-c2e4-4506-b7f2-cf37236746b9 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:37.426026 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43474b93-74f5-4455-a5fe-96fd1f6f8528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:37.426682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43474b93-74f5-4455-a5fe-96fd1f6f8528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:37.426959 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43474b93-74f5-4455-a5fe-96fd1f6f8528 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:37.427231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43474b93-74f5-4455-a5fe-96fd1f6f8528 admin admin] PATH_INFO: `/v3/users/8e2e4f64c4854e849cb282b624138d7b` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:37.428354 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43474b93-74f5-4455-a5fe-96fd1f6f8528 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e2e4f64c4854e849cb282b624138d7b)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:37.431160 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43474b93-74f5-4455-a5fe-96fd1f6f8528 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:37.506264 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67424e4a-b730-4912-9151-3974eb9591a2 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:37.507897 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-67424e4a-b730-4912-9151-3974eb9591a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:37.538452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67424e4a-b730-4912-9151-3974eb9591a2 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:37.544787 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1f7da-c49d-4738-9107-eae1a45e393f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:37.545764 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1f7da-c49d-4738-9107-eae1a45e393f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:37.546188 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1f7da-c49d-4738-9107-eae1a45e393f admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:37.546799 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1f7da-c49d-4738-9107-eae1a45e393f admin admin] PATH_INFO: `/v3/projects/d3da6026d90b427999eedc39484f26fa` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:37.549716 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1f7da-c49d-4738-9107-eae1a45e393f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3da6026d90b427999eedc39484f26fa)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:37.552368 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1f7da-c49d-4738-9107-eae1a45e393f admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:37.708398 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-43474b93-74f5-4455-a5fe-96fd1f6f8528 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e2e4f64c4854e849cb282b624138d7b'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:37.709335 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-43474b93-74f5-4455-a5fe-96fd1f6f8528 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e2e4f64c4854e849cb282b624138d7b'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:37.716869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-91b1f7da-c49d-4738-9107-eae1a45e393f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3da6026d90b427999eedc39484f26fa', 'request_id': 'req-91b1f7da-c49d-4738-9107-eae1a45e393f'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:37.808326 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-43474b93-74f5-4455-a5fe-96fd1f6f8528 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e2e4f64c4854e849cb282b624138d7b'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:37.822347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-43474b93-74f5-4455-a5fe-96fd1f6f8528 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e2e4f64c4854e849cb282b624138d7b'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:37.824024 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-43474b93-74f5-4455-a5fe-96fd1f6f8528 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e2e4f64c4854e849cb282b624138d7b'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:37.841492 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 809/1613] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:51:37 2022] DELETE /identity/v3/users/8e2e4f64c4854e849cb282b624138d7b => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:51:37.881195 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-91b1f7da-c49d-4738-9107-eae1a45e393f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3da6026d90b427999eedc39484f26fa', 'request_id': 'req-91b1f7da-c49d-4738-9107-eae1a45e393f'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:37.897461 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-91b1f7da-c49d-4738-9107-eae1a45e393f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3da6026d90b427999eedc39484f26fa', 'request_id': 'req-91b1f7da-c49d-4738-9107-eae1a45e393f'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:37.921771 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-91b1f7da-c49d-4738-9107-eae1a45e393f admin admin] The token cache is being invalidate because project d3da6026d90b427999eedc39484f26fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:37.922069 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-91b1f7da-c49d-4738-9107-eae1a45e393f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:37.923129 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-91b1f7da-c49d-4738-9107-eae1a45e393f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:37.926681 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 805/1614] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:37 2022] DELETE /identity/v3/projects/d3da6026d90b427999eedc39484f26fa => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:51:37.931843 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33233f-f4de-4d53-b96c-b981dd1a56f6 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:37.932615 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33233f-f4de-4d53-b96c-b981dd1a56f6 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:37.933143 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be33233f-f4de-4d53-b96c-b981dd1a56f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:37.968240 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-be33233f-f4de-4d53-b96c-b981dd1a56f6 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:37.968770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:37.969864 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-be33233f-f4de-4d53-b96c-b981dd1a56f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:38.031386 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 810/1615] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:38.039928 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd99cfa-027f-4ed8-bbcd-e5d184dfc2ab None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:38.040270 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd99cfa-027f-4ed8-bbcd-e5d184dfc2ab None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:38.040602 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd99cfa-027f-4ed8-bbcd-e5d184dfc2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:38.071572 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-5dd99cfa-027f-4ed8-bbcd-e5d184dfc2ab None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:38.075208 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-5dd99cfa-027f-4ed8-bbcd-e5d184dfc2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:38.095440 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 806/1616] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:38.097993 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33233f-f4de-4d53-b96c-b981dd1a56f6 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:38.109360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be33233f-f4de-4d53-b96c-b981dd1a56f6 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:38.133254 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ace574-be7c-4541-bd0f-b2bd73bd7f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:38.133947 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ace574-be7c-4541-bd0f-b2bd73bd7f46 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:38.134281 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ace574-be7c-4541-bd0f-b2bd73bd7f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:38.140571 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ace574-be7c-4541-bd0f-b2bd73bd7f46 admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:38.140571 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ace574-be7c-4541-bd0f-b2bd73bd7f46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:51:38.140571 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ace574-be7c-4541-bd0f-b2bd73bd7f46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:38.140571 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ace574-be7c-4541-bd0f-b2bd73bd7f46 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:38.155278 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 811/1617] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:51:38.162493 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd99cfa-027f-4ed8-bbcd-e5d184dfc2ab None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:38.176041 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd99cfa-027f-4ed8-bbcd-e5d184dfc2ab None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:38.182726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fca4f-7752-4040-a83b-f98ea63816bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:38.183902 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fca4f-7752-4040-a83b-f98ea63816bf admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:38.184455 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fca4f-7752-4040-a83b-f98ea63816bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:38.184742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216fca4f-7752-4040-a83b-f98ea63816bf admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:38.185386 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fca4f-7752-4040-a83b-f98ea63816bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:38.187360 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216fca4f-7752-4040-a83b-f98ea63816bf admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:38.317303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 807/1618] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:38 2022] POST /identity/v3/projects => generated 393 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:38.321158 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ace574-be7c-4541-bd0f-b2bd73bd7f46 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:38.331785 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ace574-be7c-4541-bd0f-b2bd73bd7f46 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:38.349605 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fca4f-7752-4040-a83b-f98ea63816bf admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:38.351922 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-216fca4f-7752-4040-a83b-f98ea63816bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:38.485619 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216fca4f-7752-4040-a83b-f98ea63816bf admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:38.490039 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a553f-a165-4d04-bfb3-caeac3c724e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:38.490947 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a553f-a165-4d04-bfb3-caeac3c724e0 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:38.491644 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032dccac-31be-4aa4-a763-013829350a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:38.492034 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a553f-a165-4d04-bfb3-caeac3c724e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:38.492866 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032dccac-31be-4aa4-a763-013829350a73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:38.493248 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032dccac-31be-4aa4-a763-013829350a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:38.493583 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871a553f-a165-4d04-bfb3-caeac3c724e0 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:38.493696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032dccac-31be-4aa4-a763-013829350a73 admin admin] PATH_INFO: `/v3/projects/58d521c8b35f4c86b195f70719cc453a` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:38.494939 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032dccac-31be-4aa4-a763-013829350a73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58d521c8b35f4c86b195f70719cc453a)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:38.495679 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a553f-a165-4d04-bfb3-caeac3c724e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:38.497508 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032dccac-31be-4aa4-a763-013829350a73 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:38.500675 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871a553f-a165-4d04-bfb3-caeac3c724e0 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:38.634631 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 812/1619] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:38 2022] POST /identity/v3/users => generated 297 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:38.638397 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a553f-a165-4d04-bfb3-caeac3c724e0 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:38.653229 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871a553f-a165-4d04-bfb3-caeac3c724e0 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:38.665205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-032dccac-31be-4aa4-a763-013829350a73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58d521c8b35f4c86b195f70719cc453a', 'request_id': 'req-032dccac-31be-4aa4-a763-013829350a73'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:38.747353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30dc4f2-3b10-47e7-9cb1-1f64ecb6a37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:38.748169 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30dc4f2-3b10-47e7-9cb1-1f64ecb6a37a admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:38.748393 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30dc4f2-3b10-47e7-9cb1-1f64ecb6a37a admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:38.748664 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30dc4f2-3b10-47e7-9cb1-1f64ecb6a37a admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:38.749309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30dc4f2-3b10-47e7-9cb1-1f64ecb6a37a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:38.750979 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30dc4f2-3b10-47e7-9cb1-1f64ecb6a37a admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:38.761789 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 813/1620] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:51:38.765423 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30dc4f2-3b10-47e7-9cb1-1f64ecb6a37a admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:38.776890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30dc4f2-3b10-47e7-9cb1-1f64ecb6a37a admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:38.781442 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce698c67-21be-4c63-9042-b344fabe947b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:38.782043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce698c67-21be-4c63-9042-b344fabe947b admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:38.782323 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce698c67-21be-4c63-9042-b344fabe947b admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:38.782589 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce698c67-21be-4c63-9042-b344fabe947b admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:38.784065 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce698c67-21be-4c63-9042-b344fabe947b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:38.787570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce698c67-21be-4c63-9042-b344fabe947b admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:38.798947 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 814/1621] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:38.804770 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce698c67-21be-4c63-9042-b344fabe947b admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:38.807536 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-032dccac-31be-4aa4-a763-013829350a73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58d521c8b35f4c86b195f70719cc453a', 'request_id': 'req-032dccac-31be-4aa4-a763-013829350a73'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:38.816048 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-032dccac-31be-4aa4-a763-013829350a73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58d521c8b35f4c86b195f70719cc453a', 'request_id': 'req-032dccac-31be-4aa4-a763-013829350a73'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:38.827710 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-032dccac-31be-4aa4-a763-013829350a73 admin admin] The token cache is being invalidate because project 58d521c8b35f4c86b195f70719cc453a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:38.828181 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-032dccac-31be-4aa4-a763-013829350a73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:38.828981 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-032dccac-31be-4aa4-a763-013829350a73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:38.830653 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 808/1622] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:38 2022] DELETE /identity/v3/projects/58d521c8b35f4c86b195f70719cc453a => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:51:38.836429 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0665a3-5709-4bbb-8490-f936e00f2cba None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:38.836997 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0665a3-5709-4bbb-8490-f936e00f2cba None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:38.837493 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0665a3-5709-4bbb-8490-f936e00f2cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:38.838817 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce698c67-21be-4c63-9042-b344fabe947b admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:38.850726 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431202d1-51ac-471a-a92f-98660057bbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:38.851476 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431202d1-51ac-471a-a92f-98660057bbef admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:38.851959 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431202d1-51ac-471a-a92f-98660057bbef admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:38.852380 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431202d1-51ac-471a-a92f-98660057bbef admin admin] PATH_INFO: `/v3/projects/c26d6b00c5654c71a75aaea330965bb0/users/20900e74a0cb4173a3499eff7629705a/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:38.869617 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-3c0665a3-5709-4bbb-8490-f936e00f2cba None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:38.870862 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431202d1-51ac-471a-a92f-98660057bbef admin admin] RBAC: Authorizing `identity:create_grant(project_id=c26d6b00c5654c71a75aaea330965bb0, user_id=20900e74a0cb4173a3499eff7629705a, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:38.871015 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-3c0665a3-5709-4bbb-8490-f936e00f2cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:38.887860 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431202d1-51ac-471a-a92f-98660057bbef admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:38.888006 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 809/1623] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:38.888006 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4cb9bb-753c-4b7d-aa40-73744ac056c5 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:38.889252 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4cb9bb-753c-4b7d-aa40-73744ac056c5 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:38.889252 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4cb9bb-753c-4b7d-aa40-73744ac056c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:38.909479 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-ac4cb9bb-753c-4b7d-aa40-73744ac056c5 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:38.910600 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-ac4cb9bb-753c-4b7d-aa40-73744ac056c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:38.918694 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 810/1624] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:51:38.922900 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4cb9bb-753c-4b7d-aa40-73744ac056c5 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:38.935515 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4cb9bb-753c-4b7d-aa40-73744ac056c5 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:38.940756 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f876574-5244-4790-bb63-dab21b861602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:38.944953 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f876574-5244-4790-bb63-dab21b861602 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:38.944953 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f876574-5244-4790-bb63-dab21b861602 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:38.944953 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f876574-5244-4790-bb63-dab21b861602 admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:38.944953 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f876574-5244-4790-bb63-dab21b861602 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:51:38.944953 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f876574-5244-4790-bb63-dab21b861602 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:38.944953 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f876574-5244-4790-bb63-dab21b861602 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:38.959421 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 811/1625] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:51:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:51:38.966263 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f876574-5244-4790-bb63-dab21b861602 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:38.976946 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f876574-5244-4790-bb63-dab21b861602 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:38.979794 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 815/1626] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:38 2022] PUT /identity/v3/projects/c26d6b00c5654c71a75aaea330965bb0/users/20900e74a0cb4173a3499eff7629705a/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:38.989835 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fffc489-8f0d-4c32-bc83-2d9d296817aa None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:38.990156 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fffc489-8f0d-4c32-bc83-2d9d296817aa None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:38.990434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fffc489-8f0d-4c32-bc83-2d9d296817aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:39.019661 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-7fffc489-8f0d-4c32-bc83-2d9d296817aa None None] MFA Rules not processed for user `20900e74a0cb4173a3499eff7629705a`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:39.020977 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-7fffc489-8f0d-4c32-bc83-2d9d296817aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:39.039580 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d98b102-32a0-4b6b-9c0b-167404fd3d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:39.040222 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d98b102-32a0-4b6b-9c0b-167404fd3d36 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:39.040504 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d98b102-32a0-4b6b-9c0b-167404fd3d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:39.040812 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d98b102-32a0-4b6b-9c0b-167404fd3d36 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:39.041450 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d98b102-32a0-4b6b-9c0b-167404fd3d36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:39.043333 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d98b102-32a0-4b6b-9c0b-167404fd3d36 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:39.090305 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 816/1627] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:38 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:51:39.094960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fffc489-8f0d-4c32-bc83-2d9d296817aa None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:39.096501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-7fffc489-8f0d-4c32-bc83-2d9d296817aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:39.110172 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fffc489-8f0d-4c32-bc83-2d9d296817aa None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:39.115066 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81574dab-78ce-460f-8a62-0aa9ec5616e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:39.115855 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81574dab-78ce-460f-8a62-0aa9ec5616e7 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:39.116135 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81574dab-78ce-460f-8a62-0aa9ec5616e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:39.118134 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81574dab-78ce-460f-8a62-0aa9ec5616e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:39.118134 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81574dab-78ce-460f-8a62-0aa9ec5616e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:39.118931 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81574dab-78ce-460f-8a62-0aa9ec5616e7 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:39.140198 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 812/1628] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:38 2022] POST /identity/v3/projects => generated 365 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:51:39.144776 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d98b102-32a0-4b6b-9c0b-167404fd3d36 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:39.205066 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d98b102-32a0-4b6b-9c0b-167404fd3d36 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:39.210482 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f5e64-2468-4b7c-9afb-b3708d54e301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:39.211089 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f5e64-2468-4b7c-9afb-b3708d54e301 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:39.211371 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f5e64-2468-4b7c-9afb-b3708d54e301 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:39.211647 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f5e64-2468-4b7c-9afb-b3708d54e301 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:39.212316 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f5e64-2468-4b7c-9afb-b3708d54e301 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:39.214209 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74f5e64-2468-4b7c-9afb-b3708d54e301 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:39.243798 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 817/1629] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:39 2022] POST /identity/v3/projects => generated 393 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:51:39.247801 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81574dab-78ce-460f-8a62-0aa9ec5616e7 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:39.318938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 813/1630] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:39 2022] POST /identity/v3/users => generated 283 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:51:39.322067 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81574dab-78ce-460f-8a62-0aa9ec5616e7 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:39.323130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f5e64-2468-4b7c-9afb-b3708d54e301 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:39.327839 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7912c-b8ca-44c8-8452-26247dad5035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:39.328613 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7912c-b8ca-44c8-8452-26247dad5035 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:39.329091 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7912c-b8ca-44c8-8452-26247dad5035 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:39.329369 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7912c-b8ca-44c8-8452-26247dad5035 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:39.330660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d7912c-b8ca-44c8-8452-26247dad5035 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:39.332909 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d7912c-b8ca-44c8-8452-26247dad5035 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:39.334396 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f5e64-2468-4b7c-9afb-b3708d54e301 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:39.339645 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f963eb4-4589-438e-9ad2-57fbf5afe947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:39.340400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f963eb4-4589-438e-9ad2-57fbf5afe947 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:39.340778 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f963eb4-4589-438e-9ad2-57fbf5afe947 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:39.341192 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f963eb4-4589-438e-9ad2-57fbf5afe947 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:39.341947 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f963eb4-4589-438e-9ad2-57fbf5afe947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:39.343839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f963eb4-4589-438e-9ad2-57fbf5afe947 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:39.354428 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 814/1631] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:51:39.358313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f963eb4-4589-438e-9ad2-57fbf5afe947 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:39.366749 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:39.369986 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f963eb4-4589-438e-9ad2-57fbf5afe947 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:39.374939 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38000863-e209-4438-be27-de6690df5307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:39.375521 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38000863-e209-4438-be27-de6690df5307 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:39.375831 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38000863-e209-4438-be27-de6690df5307 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:39.376102 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38000863-e209-4438-be27-de6690df5307 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:39.376843 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38000863-e209-4438-be27-de6690df5307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:39.378945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38000863-e209-4438-be27-de6690df5307 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:39.389460 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 815/1632] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:51:39.393513 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38000863-e209-4438-be27-de6690df5307 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:39.403030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38000863-e209-4438-be27-de6690df5307 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:39.407661 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1910d2-e45b-49f3-ad8c-b7e3f982ac6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:39.408312 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1910d2-e45b-49f3-ad8c-b7e3f982ac6c admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:39.408593 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1910d2-e45b-49f3-ad8c-b7e3f982ac6c admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:39.408868 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1910d2-e45b-49f3-ad8c-b7e3f982ac6c admin admin] PATH_INFO: `/v3/projects/81eced17da01430395371807a02fd180/users/d009bca4a13944c39e86b9087864e615/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:39.423063 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1910d2-e45b-49f3-ad8c-b7e3f982ac6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=81eced17da01430395371807a02fd180, user_id=d009bca4a13944c39e86b9087864e615, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:39.426208 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1910d2-e45b-49f3-ad8c-b7e3f982ac6c admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:39.441618 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 818/1633] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:39 2022] POST /identity/v3/users => generated 303 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:51:39.445514 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7912c-b8ca-44c8-8452-26247dad5035 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:39.457126 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7912c-b8ca-44c8-8452-26247dad5035 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:39.462666 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b0cbc-7588-478b-8a2b-045b69089502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:39.463559 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b0cbc-7588-478b-8a2b-045b69089502 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:39.463869 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b0cbc-7588-478b-8a2b-045b69089502 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:39.464143 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b0cbc-7588-478b-8a2b-045b69089502 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:39.473395 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b0cbc-7588-478b-8a2b-045b69089502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:39.473395 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b0cbc-7588-478b-8a2b-045b69089502 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:39.477426 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 819/1634] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:51:39.482646 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b0cbc-7588-478b-8a2b-045b69089502 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:39.493137 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b0cbc-7588-478b-8a2b-045b69089502 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:39.497792 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875bba1-fbbf-4b97-a3e1-a96eaaf6bb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:39.498514 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875bba1-fbbf-4b97-a3e1-a96eaaf6bb9e admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:39.498783 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875bba1-fbbf-4b97-a3e1-a96eaaf6bb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:39.499061 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875bba1-fbbf-4b97-a3e1-a96eaaf6bb9e admin admin] PATH_INFO: `/v3/projects/26e0a828f23a4c3da908506a798ae736/users/9d37d95148ce40769d66e7ba21100109/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:39.514960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875bba1-fbbf-4b97-a3e1-a96eaaf6bb9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=26e0a828f23a4c3da908506a798ae736, user_id=9d37d95148ce40769d66e7ba21100109, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:39.517652 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3875bba1-fbbf-4b97-a3e1-a96eaaf6bb9e admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:39.527843 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 816/1635] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:39 2022] PUT /identity/v3/projects/81eced17da01430395371807a02fd180/users/d009bca4a13944c39e86b9087864e615/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:51:39.538114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875c844-44a1-4fd5-90be-fbe2f87b0e5b None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:39.538400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875c844-44a1-4fd5-90be-fbe2f87b0e5b None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:39.538672 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875c844-44a1-4fd5-90be-fbe2f87b0e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:39.572745 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-2875c844-44a1-4fd5-90be-fbe2f87b0e5b None None] MFA Rules not processed for user `d009bca4a13944c39e86b9087864e615`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:39.573972 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-2875c844-44a1-4fd5-90be-fbe2f87b0e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:39.613335 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 820/1636] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:39 2022] PUT /identity/v3/projects/26e0a828f23a4c3da908506a798ae736/users/9d37d95148ce40769d66e7ba21100109/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:39.614397 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875bba1-fbbf-4b97-a3e1-a96eaaf6bb9e admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:39.664325 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 817/1637] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:51:39.668901 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875c844-44a1-4fd5-90be-fbe2f87b0e5b None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:39.689787 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3875bba1-fbbf-4b97-a3e1-a96eaaf6bb9e admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:39.690505 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7010a48-fba6-463f-95e8-a39d3c78af1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:39.690955 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7010a48-fba6-463f-95e8-a39d3c78af1e admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:39.691239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7010a48-fba6-463f-95e8-a39d3c78af1e admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:39.691516 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7010a48-fba6-463f-95e8-a39d3c78af1e admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:39.692381 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7010a48-fba6-463f-95e8-a39d3c78af1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:39.693853 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7010a48-fba6-463f-95e8-a39d3c78af1e admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:39.704215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 821/1638] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:39 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:51:39.709257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7010a48-fba6-463f-95e8-a39d3c78af1e admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:39.720661 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7010a48-fba6-463f-95e8-a39d3c78af1e admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:39.725983 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa4112-f8d9-441a-a40e-ad9a5b3b1790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:39.726647 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa4112-f8d9-441a-a40e-ad9a5b3b1790 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:39.726933 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa4112-f8d9-441a-a40e-ad9a5b3b1790 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:39.727214 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa4112-f8d9-441a-a40e-ad9a5b3b1790 admin admin] PATH_INFO: `/v3/projects/26e0a828f23a4c3da908506a798ae736/users/9d37d95148ce40769d66e7ba21100109/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:39.729570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa4112-f8d9-441a-a40e-ad9a5b3b1790 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26e0a828f23a4c3da908506a798ae736, user_id=9d37d95148ce40769d66e7ba21100109, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:39.732771 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa4112-f8d9-441a-a40e-ad9a5b3b1790 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:39.748387 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 822/1639] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:39 2022] PUT /identity/v3/projects/26e0a828f23a4c3da908506a798ae736/users/9d37d95148ce40769d66e7ba21100109/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:51:39.755179 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875c844-44a1-4fd5-90be-fbe2f87b0e5b None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:39.759687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca999f74-9943-4cef-83cd-72bfa4f5f132 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:39.760066 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca999f74-9943-4cef-83cd-72bfa4f5f132 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:39.760531 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca999f74-9943-4cef-83cd-72bfa4f5f132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:39.800917 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-ca999f74-9943-4cef-83cd-72bfa4f5f132 None None] MFA Rules not processed for user `9d37d95148ce40769d66e7ba21100109`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:39.804643 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-ca999f74-9943-4cef-83cd-72bfa4f5f132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:39.857496 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b97be1-ff78-4b2a-8d4e-d42ef1ed57c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:39.858635 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b97be1-ff78-4b2a-8d4e-d42ef1ed57c9 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:39.859600 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b97be1-ff78-4b2a-8d4e-d42ef1ed57c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:39.860277 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b97be1-ff78-4b2a-8d4e-d42ef1ed57c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:39.861052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b97be1-ff78-4b2a-8d4e-d42ef1ed57c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:39.864879 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b97be1-ff78-4b2a-8d4e-d42ef1ed57c9 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:39.886962 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 823/1640] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:39 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:51:39.898477 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca999f74-9943-4cef-83cd-72bfa4f5f132 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:39.899987 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-ca999f74-9943-4cef-83cd-72bfa4f5f132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:39.952656 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 818/1641] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:39 2022] POST /identity/v3/projects => generated 367 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:39.957333 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b97be1-ff78-4b2a-8d4e-d42ef1ed57c9 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:39.962658 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca999f74-9943-4cef-83cd-72bfa4f5f132 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:40.023634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc8bd0-afd1-424c-ae8d-e7cb33368c77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:40.024285 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc8bd0-afd1-424c-ae8d-e7cb33368c77 service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:40.024563 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc8bd0-afd1-424c-ae8d-e7cb33368c77 service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:40.024828 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc8bd0-afd1-424c-ae8d-e7cb33368c77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:40.065856 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b97be1-ff78-4b2a-8d4e-d42ef1ed57c9 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:40.074250 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f1f9b-4ce3-439b-a37e-45ddfb490d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:40.074965 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f1f9b-4ce3-439b-a37e-45ddfb490d10 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:40.075289 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f1f9b-4ce3-439b-a37e-45ddfb490d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:40.075569 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887f1f9b-4ce3-439b-a37e-45ddfb490d10 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:40.076287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f1f9b-4ce3-439b-a37e-45ddfb490d10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:40.078253 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887f1f9b-4ce3-439b-a37e-45ddfb490d10 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:40.111310 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc8bd0-afd1-424c-ae8d-e7cb33368c77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:40.112818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc8bd0-afd1-424c-ae8d-e7cb33368c77 service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:40.122403 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 824/1642] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:39 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:40.164605 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:40.169570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc8bd0-afd1-424c-ae8d-e7cb33368c77 service cinder] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:40.170749 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 819/1643] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:39 2022] POST /identity/v3/users => generated 290 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:51:40.170994 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-82bc8bd0-afd1-424c-ae8d-e7cb33368c77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:40.174652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f1f9b-4ce3-439b-a37e-45ddfb490d10 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:40.181857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc8bd0-afd1-424c-ae8d-e7cb33368c77 service cinder] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:40.184249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887f1f9b-4ce3-439b-a37e-45ddfb490d10 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:40.185796 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5ec74-b090-4e6a-a817-65ad664621cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:40.186404 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5ec74-b090-4e6a-a817-65ad664621cb service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:40.186681 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5ec74-b090-4e6a-a817-65ad664621cb service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:40.186951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5ec74-b090-4e6a-a817-65ad664621cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:40.188910 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ea040-3f50-4a19-b5be-b8473a2a0486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:40.189505 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ea040-3f50-4a19-b5be-b8473a2a0486 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:40.189784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ea040-3f50-4a19-b5be-b8473a2a0486 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:40.190060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2ea040-3f50-4a19-b5be-b8473a2a0486 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:40.190720 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2ea040-3f50-4a19-b5be-b8473a2a0486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:40.192407 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2ea040-3f50-4a19-b5be-b8473a2a0486 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:40.202175 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 820/1644] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:51:40.207089 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ea040-3f50-4a19-b5be-b8473a2a0486 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:40.217787 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2ea040-3f50-4a19-b5be-b8473a2a0486 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:40.222238 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b849ff-9435-45c1-a9fe-6ef8185498eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:40.222915 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b849ff-9435-45c1-a9fe-6ef8185498eb admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:40.223179 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b849ff-9435-45c1-a9fe-6ef8185498eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:40.223459 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b849ff-9435-45c1-a9fe-6ef8185498eb admin admin] PATH_INFO: `/v3/projects/e1ce34cf5ce14d34a5d9ef74792615a3/users/a686fd3aa14243c5963d0b2913beb941/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:40.238341 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b849ff-9435-45c1-a9fe-6ef8185498eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1ce34cf5ce14d34a5d9ef74792615a3, user_id=a686fd3aa14243c5963d0b2913beb941, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:40.241021 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b849ff-9435-45c1-a9fe-6ef8185498eb admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:40.245671 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5ec74-b090-4e6a-a817-65ad664621cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:40.247290 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5ec74-b090-4e6a-a817-65ad664621cb service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:40.256416 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 825/1645] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:40 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:51:40.333545 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 821/1646] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:40 2022] PUT /identity/v3/projects/e1ce34cf5ce14d34a5d9ef74792615a3/users/a686fd3aa14243c5963d0b2913beb941/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:40.337610 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5ec74-b090-4e6a-a817-65ad664621cb service cinder] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:40.350578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5ec74-b090-4e6a-a817-65ad664621cb service cinder] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:40.414455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40f7f61-4fd3-4bdb-8989-32aced5aee0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:40.415072 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f7f61-4fd3-4bdb-8989-32aced5aee0a admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:40.415356 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f7f61-4fd3-4bdb-8989-32aced5aee0a admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:40.415640 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40f7f61-4fd3-4bdb-8989-32aced5aee0a admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:40.416333 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40f7f61-4fd3-4bdb-8989-32aced5aee0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:40.417929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40f7f61-4fd3-4bdb-8989-32aced5aee0a admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:40.427489 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 826/1647] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:40.431656 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b849ff-9435-45c1-a9fe-6ef8185498eb admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:40.442339 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b849ff-9435-45c1-a9fe-6ef8185498eb admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:40.447167 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f893b01d-7989-4e87-925c-db15bb817d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:40.447873 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893b01d-7989-4e87-925c-db15bb817d72 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:40.450509 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893b01d-7989-4e87-925c-db15bb817d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:40.450509 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f893b01d-7989-4e87-925c-db15bb817d72 admin admin] PATH_INFO: `/v3/projects/e1ce34cf5ce14d34a5d9ef74792615a3/users/a686fd3aa14243c5963d0b2913beb941/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:40.450509 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f893b01d-7989-4e87-925c-db15bb817d72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1ce34cf5ce14d34a5d9ef74792615a3, user_id=a686fd3aa14243c5963d0b2913beb941, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:40.453010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f893b01d-7989-4e87-925c-db15bb817d72 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:40.463838 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 822/1648] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:40 2022] PUT /identity/v3/projects/e1ce34cf5ce14d34a5d9ef74792615a3/users/a686fd3aa14243c5963d0b2913beb941/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:40.474789 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf19e21-2126-46ee-8dfc-3ed49daa5f01 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:40.475025 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf19e21-2126-46ee-8dfc-3ed49daa5f01 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:40.475303 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf19e21-2126-46ee-8dfc-3ed49daa5f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:40.504240 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-1bf19e21-2126-46ee-8dfc-3ed49daa5f01 None None] MFA Rules not processed for user `a686fd3aa14243c5963d0b2913beb941`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:40.505470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-1bf19e21-2126-46ee-8dfc-3ed49daa5f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:40.572559 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 827/1649] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:40 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:51:40.585289 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f893b01d-7989-4e87-925c-db15bb817d72 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:40.608109 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:40.641069 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f893b01d-7989-4e87-925c-db15bb817d72 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:40.645481 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d8320-3f04-4446-a962-5210b04c34ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:40.646096 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d8320-3f04-4446-a962-5210b04c34ee service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:40.647153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d8320-3f04-4446-a962-5210b04c34ee service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:40.647153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d8320-3f04-4446-a962-5210b04c34ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:40.657527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d8320-3f04-4446-a962-5210b04c34ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:40.659043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d8320-3f04-4446-a962-5210b04c34ee service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:40.669042 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 823/1650] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:40 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:40.848579 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:40.890140 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf19e21-2126-46ee-8dfc-3ed49daa5f01 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:40.901294 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf19e21-2126-46ee-8dfc-3ed49daa5f01 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:40.905125 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d81f9-15de-4be2-bb42-75fc45517c96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:40.905886 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d81f9-15de-4be2-bb42-75fc45517c96 service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:40.906231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d81f9-15de-4be2-bb42-75fc45517c96 service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:40.906674 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9d81f9-15de-4be2-bb42-75fc45517c96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:40.975044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d81f9-15de-4be2-bb42-75fc45517c96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:40.976612 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9d81f9-15de-4be2-bb42-75fc45517c96 service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:40.986530 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 828/1651] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:41.070982 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:41.561203 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=77,keystone:DELETE=20,keystone:INSERT=3 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:41.980580 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:42.306355 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d8320-3f04-4446-a962-5210b04c34ee service cinder] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:42.317057 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d8320-3f04-4446-a962-5210b04c34ee service cinder] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:42.321037 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd777d9-f31b-4a65-a491-f0e01d1954e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:42.321650 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd777d9-f31b-4a65-a491-f0e01d1954e1 service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:42.321931 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd777d9-f31b-4a65-a491-f0e01d1954e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:42.322205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd777d9-f31b-4a65-a491-f0e01d1954e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:42.324111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-abd777d9-f31b-4a65-a491-f0e01d1954e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:42.401269 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd777d9-f31b-4a65-a491-f0e01d1954e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:42.402749 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd777d9-f31b-4a65-a491-f0e01d1954e1 service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:42.412517 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 824/1652] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:42 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:42.648017 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:42.751365 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:43.076980 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:43.161262 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:43.648346 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:43.743715 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:46.895138 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d81f9-15de-4be2-bb42-75fc45517c96 service cinder] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:46.896802 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-8f9d81f9-15de-4be2-bb42-75fc45517c96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:46.993056 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:47.026956 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9d81f9-15de-4be2-bb42-75fc45517c96 service cinder] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:47.039349 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ef378-fa44-401c-8de5-f8d5d97a9a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:47.040095 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ef378-fa44-401c-8de5-f8d5d97a9a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:47.041703 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ef378-fa44-401c-8de5-f8d5d97a9a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:47.041703 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ef378-fa44-401c-8de5-f8d5d97a9a4d admin admin] PATH_INFO: `/v3/users/0ea7d079331240e2858d5f546490c6f5` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:47.044891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0ef378-fa44-401c-8de5-f8d5d97a9a4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ea7d079331240e2858d5f546490c6f5)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:47.052513 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0ef378-fa44-401c-8de5-f8d5d97a9a4d admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:47.147027 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd777d9-f31b-4a65-a491-f0e01d1954e1 service cinder] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:47.153137 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-abd777d9-f31b-4a65-a491-f0e01d1954e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:47.167155 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd777d9-f31b-4a65-a491-f0e01d1954e1 service cinder] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:47.178938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbaf4cc-32cd-4392-aac7-029361326eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:47.179901 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbaf4cc-32cd-4392-aac7-029361326eff admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:47.180504 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbaf4cc-32cd-4392-aac7-029361326eff admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:47.180787 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbaf4cc-32cd-4392-aac7-029361326eff admin admin] PATH_INFO: `/v3/users/32b8473a2d0540ff842f0a9fede4a38e` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:47.184304 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbaf4cc-32cd-4392-aac7-029361326eff admin admin] RBAC: Authorizing `identity:delete_user(user_id=32b8473a2d0540ff842f0a9fede4a38e)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:47.186237 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbaf4cc-32cd-4392-aac7-029361326eff admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:47.277066 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-8f0ef378-fa44-401c-8de5-f8d5d97a9a4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ea7d079331240e2858d5f546490c6f5'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:47.278065 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-8f0ef378-fa44-401c-8de5-f8d5d97a9a4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ea7d079331240e2858d5f546490c6f5'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:47.329343 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-8f0ef378-fa44-401c-8de5-f8d5d97a9a4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ea7d079331240e2858d5f546490c6f5'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:47.366719 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-8f0ef378-fa44-401c-8de5-f8d5d97a9a4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ea7d079331240e2858d5f546490c6f5'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:47.367597 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-8f0ef378-fa44-401c-8de5-f8d5d97a9a4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ea7d079331240e2858d5f546490c6f5'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:47.401133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-1bbaf4cc-32cd-4392-aac7-029361326eff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32b8473a2d0540ff842f0a9fede4a38e'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:47.405125 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 829/1653] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:51:46 2022] DELETE /identity/v3/users/0ea7d079331240e2858d5f546490c6f5 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:47.405125 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ef378-fa44-401c-8de5-f8d5d97a9a4d admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:47.405562 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-8f0ef378-fa44-401c-8de5-f8d5d97a9a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:47.415098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:47.518066 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ef378-fa44-401c-8de5-f8d5d97a9a4d admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:47.523940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766ebfb-e934-4b77-a6c7-ef73ab8f0f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:47.524589 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766ebfb-e934-4b77-a6c7-ef73ab8f0f2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:47.524865 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766ebfb-e934-4b77-a6c7-ef73ab8f0f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:47.525152 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3766ebfb-e934-4b77-a6c7-ef73ab8f0f2f admin admin] PATH_INFO: `/v3/users/20900e74a0cb4173a3499eff7629705a` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:47.526380 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766ebfb-e934-4b77-a6c7-ef73ab8f0f2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=20900e74a0cb4173a3499eff7629705a)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:47.528354 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3766ebfb-e934-4b77-a6c7-ef73ab8f0f2f admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:47.528980 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-1bbaf4cc-32cd-4392-aac7-029361326eff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32b8473a2d0540ff842f0a9fede4a38e'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:47.549140 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-1bbaf4cc-32cd-4392-aac7-029361326eff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32b8473a2d0540ff842f0a9fede4a38e'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:47.549924 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-1bbaf4cc-32cd-4392-aac7-029361326eff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32b8473a2d0540ff842f0a9fede4a38e'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:47.550567 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-1bbaf4cc-32cd-4392-aac7-029361326eff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32b8473a2d0540ff842f0a9fede4a38e'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:47.566032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 825/1654] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:51:47 2022] DELETE /identity/v3/users/32b8473a2d0540ff842f0a9fede4a38e => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:47.567093 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ae59f-7f42-44ab-bbcc-b06edcfa716e None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:47.567379 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ae59f-7f42-44ab-bbcc-b06edcfa716e None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:47.567972 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ae59f-7f42-44ab-bbcc-b06edcfa716e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:47.590116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-297ae59f-7f42-44ab-bbcc-b06edcfa716e None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:47.591473 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-297ae59f-7f42-44ab-bbcc-b06edcfa716e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:47.678361 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 826/1655] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:47.678361 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0cb6f-2cb2-4d82-88dc-ea77519a1cef None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:47.682675 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0cb6f-2cb2-4d82-88dc-ea77519a1cef None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:47.682675 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0cb6f-2cb2-4d82-88dc-ea77519a1cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:47.712654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-51e0cb6f-2cb2-4d82-88dc-ea77519a1cef None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:47.713916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-51e0cb6f-2cb2-4d82-88dc-ea77519a1cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:47.726996 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 827/1656] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:47.760092 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0cb6f-2cb2-4d82-88dc-ea77519a1cef None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:47.762240 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-51e0cb6f-2cb2-4d82-88dc-ea77519a1cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:47.765235 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3766ebfb-e934-4b77-a6c7-ef73ab8f0f2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20900e74a0cb4173a3499eff7629705a'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:47.766323 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3766ebfb-e934-4b77-a6c7-ef73ab8f0f2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20900e74a0cb4173a3499eff7629705a'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:47.869302 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3766ebfb-e934-4b77-a6c7-ef73ab8f0f2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20900e74a0cb4173a3499eff7629705a'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:47.871821 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0cb6f-2cb2-4d82-88dc-ea77519a1cef None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:47.877090 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb9840-bed3-4394-aff5-e31672ba4eab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:47.878110 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb9840-bed3-4394-aff5-e31672ba4eab service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:47.880027 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb9840-bed3-4394-aff5-e31672ba4eab service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:47.880027 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb9840-bed3-4394-aff5-e31672ba4eab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:47.882688 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:47.887586 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-f4fb9840-bed3-4394-aff5-e31672ba4eab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:47.898143 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3766ebfb-e934-4b77-a6c7-ef73ab8f0f2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20900e74a0cb4173a3499eff7629705a'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:47.899047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3766ebfb-e934-4b77-a6c7-ef73ab8f0f2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20900e74a0cb4173a3499eff7629705a'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:47.916930 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 830/1657] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:51:47 2022] DELETE /identity/v3/users/20900e74a0cb4173a3499eff7629705a => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:47.918107 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766ebfb-e934-4b77-a6c7-ef73ab8f0f2f admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:47.918856 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb9840-bed3-4394-aff5-e31672ba4eab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:47.920405 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb9840-bed3-4394-aff5-e31672ba4eab service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:47.920758 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-3766ebfb-e934-4b77-a6c7-ef73ab8f0f2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:47.940354 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 828/1658] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:51:47.942563 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae7060-be87-49b3-943a-d9d28bf9e986 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:47.942563 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae7060-be87-49b3-943a-d9d28bf9e986 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:47.942737 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae7060-be87-49b3-943a-d9d28bf9e986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:47.992430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-b9ae7060-be87-49b3-943a-d9d28bf9e986 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:47.993662 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-b9ae7060-be87-49b3-943a-d9d28bf9e986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:48.006915 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3766ebfb-e934-4b77-a6c7-ef73ab8f0f2f admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:48.011715 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05577a7-1ead-4829-8f1a-b7ac4bd4da52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:48.012500 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05577a7-1ead-4829-8f1a-b7ac4bd4da52 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:48.013154 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05577a7-1ead-4829-8f1a-b7ac4bd4da52 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:48.013440 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05577a7-1ead-4829-8f1a-b7ac4bd4da52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:48.015232 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-a05577a7-1ead-4829-8f1a-b7ac4bd4da52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:48.097183 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 829/1659] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:48.118694 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae7060-be87-49b3-943a-d9d28bf9e986 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:48.122333 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05577a7-1ead-4829-8f1a-b7ac4bd4da52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:48.123901 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05577a7-1ead-4829-8f1a-b7ac4bd4da52 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:48.132296 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae7060-be87-49b3-943a-d9d28bf9e986 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:48.136439 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8b05d-16db-497a-9087-bdc3c18293c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:48.137278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8b05d-16db-497a-9087-bdc3c18293c3 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:48.137633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8b05d-16db-497a-9087-bdc3c18293c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:48.137979 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8b05d-16db-497a-9087-bdc3c18293c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:48.145670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:48.148156 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 831/1660] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:48.150699 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8b05d-16db-497a-9087-bdc3c18293c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:48.152416 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8b05d-16db-497a-9087-bdc3c18293c3 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:48.164751 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 830/1661] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:51:48.960610 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05577a7-1ead-4829-8f1a-b7ac4bd4da52 service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:48.962061 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-a05577a7-1ead-4829-8f1a-b7ac4bd4da52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:48.974540 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05577a7-1ead-4829-8f1a-b7ac4bd4da52 service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:48.979457 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948c1ba-c80c-4c66-9432-a666941b3c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:48.980136 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948c1ba-c80c-4c66-9432-a666941b3c40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:48.980420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948c1ba-c80c-4c66-9432-a666941b3c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:48.980878 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948c1ba-c80c-4c66-9432-a666941b3c40 admin admin] PATH_INFO: `/v3/users/d009bca4a13944c39e86b9087864e615` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:48.981994 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948c1ba-c80c-4c66-9432-a666941b3c40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d009bca4a13944c39e86b9087864e615)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:48.984432 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948c1ba-c80c-4c66-9432-a666941b3c40 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:49.157263 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d948c1ba-c80c-4c66-9432-a666941b3c40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd009bca4a13944c39e86b9087864e615'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:49.158251 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d948c1ba-c80c-4c66-9432-a666941b3c40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd009bca4a13944c39e86b9087864e615'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:49.204779 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8b05d-16db-497a-9087-bdc3c18293c3 service neutron] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:49.207765 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-7ed8b05d-16db-497a-9087-bdc3c18293c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:49.223311 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8b05d-16db-497a-9087-bdc3c18293c3 service neutron] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:49.228907 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec2da1-64a0-472e-868c-318675645ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:49.230327 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec2da1-64a0-472e-868c-318675645ee9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:49.230886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec2da1-64a0-472e-868c-318675645ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:49.231375 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec2da1-64a0-472e-868c-318675645ee9 admin admin] PATH_INFO: `/v3/projects/e1bf4b9a5bab4c6ea3c9c24eb93a909f` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:49.232919 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec2da1-64a0-472e-868c-318675645ee9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1bf4b9a5bab4c6ea3c9c24eb93a909f)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:49.235185 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec2da1-64a0-472e-868c-318675645ee9 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:49.242778 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d948c1ba-c80c-4c66-9432-a666941b3c40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd009bca4a13944c39e86b9087864e615'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:49.265789 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d948c1ba-c80c-4c66-9432-a666941b3c40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd009bca4a13944c39e86b9087864e615'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:49.267884 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d948c1ba-c80c-4c66-9432-a666941b3c40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd009bca4a13944c39e86b9087864e615'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:49.305366 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 832/1662] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:51:48 2022] DELETE /identity/v3/users/d009bca4a13944c39e86b9087864e615 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:49.309681 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420093c-ea73-4f50-8584-93e44a4650ad None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:49.310108 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420093c-ea73-4f50-8584-93e44a4650ad None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:49.310579 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420093c-ea73-4f50-8584-93e44a4650ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:49.319321 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:49.330326 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-8420093c-ea73-4f50-8584-93e44a4650ad None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:49.332402 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-8420093c-ea73-4f50-8584-93e44a4650ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:49.384440 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-efec2da1-64a0-472e-868c-318675645ee9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1bf4b9a5bab4c6ea3c9c24eb93a909f', 'request_id': 'req-efec2da1-64a0-472e-868c-318675645ee9'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:49.404672 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 833/1663] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:51:49.405455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420093c-ea73-4f50-8584-93e44a4650ad None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:49.406902 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-8420093c-ea73-4f50-8584-93e44a4650ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:49.420355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420093c-ea73-4f50-8584-93e44a4650ad None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:49.476161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-efec2da1-64a0-472e-868c-318675645ee9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1bf4b9a5bab4c6ea3c9c24eb93a909f', 'request_id': 'req-efec2da1-64a0-472e-868c-318675645ee9'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:49.486557 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-efec2da1-64a0-472e-868c-318675645ee9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1bf4b9a5bab4c6ea3c9c24eb93a909f', 'request_id': 'req-efec2da1-64a0-472e-868c-318675645ee9'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:49.497443 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2afac-ee55-4cdd-a432-e2f7eab96180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:49.498096 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2afac-ee55-4cdd-a432-e2f7eab96180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:49.498377 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2afac-ee55-4cdd-a432-e2f7eab96180 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:49.498653 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2afac-ee55-4cdd-a432-e2f7eab96180 admin admin] PATH_INFO: `/v3/projects/c0e74d2edc8540868fd10f0efd97b9e9` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:49.499835 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c2afac-ee55-4cdd-a432-e2f7eab96180 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0e74d2edc8540868fd10f0efd97b9e9)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:49.501761 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c2afac-ee55-4cdd-a432-e2f7eab96180 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:49.502695 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-efec2da1-64a0-472e-868c-318675645ee9 admin admin] The token cache is being invalidate because project e1bf4b9a5bab4c6ea3c9c24eb93a909f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:49.503844 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-efec2da1-64a0-472e-868c-318675645ee9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:49.504837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-efec2da1-64a0-472e-868c-318675645ee9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:49.511253 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 831/1664] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:49 2022] DELETE /identity/v3/projects/e1bf4b9a5bab4c6ea3c9c24eb93a909f => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:51:49.520984 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec2da1-64a0-472e-868c-318675645ee9 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:49.524300 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-efec2da1-64a0-472e-868c-318675645ee9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:49.600811 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec2da1-64a0-472e-868c-318675645ee9 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:49.624390 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-12c2afac-ee55-4cdd-a432-e2f7eab96180 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0e74d2edc8540868fd10f0efd97b9e9', 'request_id': 'req-12c2afac-ee55-4cdd-a432-e2f7eab96180'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:49.688035 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4bbe3c-b132-49cf-a070-da1cfc5b8659 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:49.688723 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bbe3c-b132-49cf-a070-da1cfc5b8659 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:49.689149 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bbe3c-b132-49cf-a070-da1cfc5b8659 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:49.689428 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bbe3c-b132-49cf-a070-da1cfc5b8659 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:49.691979 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-3e4bbe3c-b132-49cf-a070-da1cfc5b8659 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:49.739633 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-12c2afac-ee55-4cdd-a432-e2f7eab96180 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0e74d2edc8540868fd10f0efd97b9e9', 'request_id': 'req-12c2afac-ee55-4cdd-a432-e2f7eab96180'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:49.745987 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:49.754192 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-12c2afac-ee55-4cdd-a432-e2f7eab96180 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0e74d2edc8540868fd10f0efd97b9e9', 'request_id': 'req-12c2afac-ee55-4cdd-a432-e2f7eab96180'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:49.769401 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-12c2afac-ee55-4cdd-a432-e2f7eab96180 admin admin] The token cache is being invalidate because project c0e74d2edc8540868fd10f0efd97b9e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:49.769613 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-12c2afac-ee55-4cdd-a432-e2f7eab96180 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:49.770367 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-12c2afac-ee55-4cdd-a432-e2f7eab96180 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:49.773012 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 834/1665] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:49 2022] DELETE /identity/v3/projects/c0e74d2edc8540868fd10f0efd97b9e9 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:49.773920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2afac-ee55-4cdd-a432-e2f7eab96180 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:49.790478 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-12c2afac-ee55-4cdd-a432-e2f7eab96180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:49.790740 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4bbe3c-b132-49cf-a070-da1cfc5b8659 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:49.790740 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4bbe3c-b132-49cf-a070-da1cfc5b8659 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:49.790880 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2afac-ee55-4cdd-a432-e2f7eab96180 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:49.796691 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 832/1666] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:49.798433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4bbe3c-b132-49cf-a070-da1cfc5b8659 service neutron] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:49.801356 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ae7600-9e72-47ba-8224-0b013db6248d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:49.801702 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae7600-9e72-47ba-8224-0b013db6248d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:49.801983 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae7600-9e72-47ba-8224-0b013db6248d admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:49.802257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae7600-9e72-47ba-8224-0b013db6248d admin admin] PATH_INFO: `/v3/projects/c26d6b00c5654c71a75aaea330965bb0` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:49.803526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-3e4bbe3c-b132-49cf-a070-da1cfc5b8659 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:49.808303 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ae7600-9e72-47ba-8224-0b013db6248d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c26d6b00c5654c71a75aaea330965bb0)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:49.810375 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ae7600-9e72-47ba-8224-0b013db6248d admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:49.825587 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4bbe3c-b132-49cf-a070-da1cfc5b8659 service neutron] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:49.831519 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb09974-f938-4716-a641-f6fc51f2dfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:49.832342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb09974-f938-4716-a641-f6fc51f2dfdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:49.832632 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb09974-f938-4716-a641-f6fc51f2dfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:49.832908 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb09974-f938-4716-a641-f6fc51f2dfdc admin admin] PATH_INFO: `/v3/users/f0dc9ba6b8cc477badeeb078c883e292` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:49.834104 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb09974-f938-4716-a641-f6fc51f2dfdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0dc9ba6b8cc477badeeb078c883e292)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:49.836028 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb09974-f938-4716-a641-f6fc51f2dfdc admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:49.957687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:49.981087 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d9ae7600-9e72-47ba-8224-0b013db6248d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c26d6b00c5654c71a75aaea330965bb0', 'request_id': 'req-d9ae7600-9e72-47ba-8224-0b013db6248d'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:50.047795 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d9ae7600-9e72-47ba-8224-0b013db6248d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c26d6b00c5654c71a75aaea330965bb0', 'request_id': 'req-d9ae7600-9e72-47ba-8224-0b013db6248d'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:50.059717 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d9ae7600-9e72-47ba-8224-0b013db6248d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c26d6b00c5654c71a75aaea330965bb0', 'request_id': 'req-d9ae7600-9e72-47ba-8224-0b013db6248d'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:50.068924 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-4cb09974-f938-4716-a641-f6fc51f2dfdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0dc9ba6b8cc477badeeb078c883e292'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:50.074862 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d9ae7600-9e72-47ba-8224-0b013db6248d admin admin] The token cache is being invalidate because project c26d6b00c5654c71a75aaea330965bb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:50.075390 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d9ae7600-9e72-47ba-8224-0b013db6248d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:50.076261 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d9ae7600-9e72-47ba-8224-0b013db6248d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:50.078053 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 835/1667] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:49 2022] DELETE /identity/v3/projects/c26d6b00c5654c71a75aaea330965bb0 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:50.083272 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684614b-ae0c-487e-83d3-9bc449b7cbcd None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:50.083823 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684614b-ae0c-487e-83d3-9bc449b7cbcd None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:50.084286 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684614b-ae0c-487e-83d3-9bc449b7cbcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:50.108841 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-8684614b-ae0c-487e-83d3-9bc449b7cbcd None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:50.110461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-8684614b-ae0c-487e-83d3-9bc449b7cbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:50.167779 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-4cb09974-f938-4716-a641-f6fc51f2dfdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0dc9ba6b8cc477badeeb078c883e292'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:50.188470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 836/1668] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:50.189924 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684614b-ae0c-487e-83d3-9bc449b7cbcd None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:50.192823 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-8684614b-ae0c-487e-83d3-9bc449b7cbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:50.199025 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-4cb09974-f938-4716-a641-f6fc51f2dfdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0dc9ba6b8cc477badeeb078c883e292'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:50.200019 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-4cb09974-f938-4716-a641-f6fc51f2dfdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0dc9ba6b8cc477badeeb078c883e292'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:50.200643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-4cb09974-f938-4716-a641-f6fc51f2dfdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0dc9ba6b8cc477badeeb078c883e292'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:50.219658 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684614b-ae0c-487e-83d3-9bc449b7cbcd None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:50.225476 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f790801-6bd8-4231-873e-125c79e91326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:50.226369 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f790801-6bd8-4231-873e-125c79e91326 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:50.226822 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f790801-6bd8-4231-873e-125c79e91326 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:50.227422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f790801-6bd8-4231-873e-125c79e91326 admin admin] PATH_INFO: `/v3/users/9d37d95148ce40769d66e7ba21100109` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:50.229986 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f790801-6bd8-4231-873e-125c79e91326 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d37d95148ce40769d66e7ba21100109)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:50.235035 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f790801-6bd8-4231-873e-125c79e91326 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:50.239465 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 833/1669] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:51:49 2022] DELETE /identity/v3/users/f0dc9ba6b8cc477badeeb078c883e292 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:50.241187 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260dfc1-2405-4052-97c8-486e41d47056 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:50.244258 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260dfc1-2405-4052-97c8-486e41d47056 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:50.244686 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260dfc1-2405-4052-97c8-486e41d47056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:50.284580 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-6260dfc1-2405-4052-97c8-486e41d47056 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:50.286076 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-6260dfc1-2405-4052-97c8-486e41d47056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:50.374856 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 834/1670] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:50.380120 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6260dfc1-2405-4052-97c8-486e41d47056 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:50.402338 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6260dfc1-2405-4052-97c8-486e41d47056 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:50.411088 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd62659-cc47-4442-ab37-a42194f826af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:50.412436 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd62659-cc47-4442-ab37-a42194f826af admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:50.412898 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd62659-cc47-4442-ab37-a42194f826af admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:50.413314 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd62659-cc47-4442-ab37-a42194f826af admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:50.414354 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd62659-cc47-4442-ab37-a42194f826af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:51:50.415071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd62659-cc47-4442-ab37-a42194f826af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:50.417418 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd62659-cc47-4442-ab37-a42194f826af admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:50.436017 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 835/1671] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:51:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:51:50.441578 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd62659-cc47-4442-ab37-a42194f826af admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:50.455235 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd62659-cc47-4442-ab37-a42194f826af admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:50.464368 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e20e9f-5c06-4255-8663-b651a8e646ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:50.465339 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e20e9f-5c06-4255-8663-b651a8e646ab admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:50.465764 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e20e9f-5c06-4255-8663-b651a8e646ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:50.469089 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e20e9f-5c06-4255-8663-b651a8e646ab admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:50.469089 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e20e9f-5c06-4255-8663-b651a8e646ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:50.471113 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e20e9f-5c06-4255-8663-b651a8e646ab admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:50.524313 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-6f790801-6bd8-4231-873e-125c79e91326 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d37d95148ce40769d66e7ba21100109'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:50.527492 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-6f790801-6bd8-4231-873e-125c79e91326 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d37d95148ce40769d66e7ba21100109'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:50.641937 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 836/1672] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:50 2022] POST /identity/v3/projects => generated 359 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:50.643155 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e20e9f-5c06-4255-8663-b651a8e646ab admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:50.644934 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-d4e20e9f-5c06-4255-8663-b651a8e646ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:50.668355 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:50.695967 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-6f790801-6bd8-4231-873e-125c79e91326 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d37d95148ce40769d66e7ba21100109'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:50.752972 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-6f790801-6bd8-4231-873e-125c79e91326 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d37d95148ce40769d66e7ba21100109'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:50.752972 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-6f790801-6bd8-4231-873e-125c79e91326 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d37d95148ce40769d66e7ba21100109'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:50.753190 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e20e9f-5c06-4255-8663-b651a8e646ab admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:50.770426 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3c74f-d1d5-4478-995e-2974314b679b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:50.770879 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3c74f-d1d5-4478-995e-2974314b679b admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:50.771457 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3c74f-d1d5-4478-995e-2974314b679b admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:50.771866 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d3c74f-d1d5-4478-995e-2974314b679b admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:50.780708 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d3c74f-d1d5-4478-995e-2974314b679b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:50.780708 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d3c74f-d1d5-4478-995e-2974314b679b admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:50.800789 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 837/1673] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:51:50 2022] DELETE /identity/v3/users/9d37d95148ce40769d66e7ba21100109 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:50.803920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f790801-6bd8-4231-873e-125c79e91326 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:50.808217 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6f790801-6bd8-4231-873e-125c79e91326 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:50.875508 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:50.904679 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 837/1674] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:50 2022] POST /identity/v3/users => generated 280 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:50.913496 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3c74f-d1d5-4478-995e-2974314b679b admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:50.940824 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d3c74f-d1d5-4478-995e-2974314b679b admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:50.947316 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac90b4c-bb14-485a-ba6c-4ca5efb0d5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:50.948009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f790801-6bd8-4231-873e-125c79e91326 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:50.948333 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac90b4c-bb14-485a-ba6c-4ca5efb0d5d7 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:50.948759 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac90b4c-bb14-485a-ba6c-4ca5efb0d5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:50.949201 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac90b4c-bb14-485a-ba6c-4ca5efb0d5d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:50.950014 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac90b4c-bb14-485a-ba6c-4ca5efb0d5d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:50.952488 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac90b4c-bb14-485a-ba6c-4ca5efb0d5d7 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:50.953390 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ab58d-bd91-4c9e-8717-86cd91bde874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:50.954049 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ab58d-bd91-4c9e-8717-86cd91bde874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:50.954368 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ab58d-bd91-4c9e-8717-86cd91bde874 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:50.954669 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ab58d-bd91-4c9e-8717-86cd91bde874 admin admin] PATH_INFO: `/v3/projects/81eced17da01430395371807a02fd180` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:50.961878 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ab58d-bd91-4c9e-8717-86cd91bde874 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81eced17da01430395371807a02fd180)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:50.964191 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ab58d-bd91-4c9e-8717-86cd91bde874 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:50.965525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 838/1675] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:50.973330 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac90b4c-bb14-485a-ba6c-4ca5efb0d5d7 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:50.990001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac90b4c-bb14-485a-ba6c-4ca5efb0d5d7 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:50.995993 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7d5e5-11cc-4893-ab09-d266739c5a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:50.997234 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7d5e5-11cc-4893-ab09-d266739c5a83 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:50.997832 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7d5e5-11cc-4893-ab09-d266739c5a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:50.997951 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a7d5e5-11cc-4893-ab09-d266739c5a83 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:50.999319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a7d5e5-11cc-4893-ab09-d266739c5a83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:51.000980 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a7d5e5-11cc-4893-ab09-d266739c5a83 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:51.035374 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 839/1676] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:51.040871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7d5e5-11cc-4893-ab09-d266739c5a83 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:51.063335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a7d5e5-11cc-4893-ab09-d266739c5a83 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:51.098661 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77bab36-3ad6-4cab-b204-7d151df2948b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:51.099343 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77bab36-3ad6-4cab-b204-7d151df2948b admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:51.099623 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77bab36-3ad6-4cab-b204-7d151df2948b admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:51.099928 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77bab36-3ad6-4cab-b204-7d151df2948b admin admin] PATH_INFO: `/v3/projects/ec18b195725f4467bcd946fe679f1ad0/users/0a901f6717074380a9c673fcaf2f52b6/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:51.115408 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-b49ab58d-bd91-4c9e-8717-86cd91bde874 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81eced17da01430395371807a02fd180', 'request_id': 'req-b49ab58d-bd91-4c9e-8717-86cd91bde874'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:51.128397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77bab36-3ad6-4cab-b204-7d151df2948b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec18b195725f4467bcd946fe679f1ad0, user_id=0a901f6717074380a9c673fcaf2f52b6, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:51.132163 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77bab36-3ad6-4cab-b204-7d151df2948b admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:51.201865 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-b49ab58d-bd91-4c9e-8717-86cd91bde874 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81eced17da01430395371807a02fd180', 'request_id': 'req-b49ab58d-bd91-4c9e-8717-86cd91bde874'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:51.208770 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-b49ab58d-bd91-4c9e-8717-86cd91bde874 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81eced17da01430395371807a02fd180', 'request_id': 'req-b49ab58d-bd91-4c9e-8717-86cd91bde874'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:51.209448 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 840/1677] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:51 2022] PUT /identity/v3/projects/ec18b195725f4467bcd946fe679f1ad0/users/0a901f6717074380a9c673fcaf2f52b6/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:51.210215 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77bab36-3ad6-4cab-b204-7d151df2948b admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:51.211994 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-a77bab36-3ad6-4cab-b204-7d151df2948b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:51.248377 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-b49ab58d-bd91-4c9e-8717-86cd91bde874 admin admin] The token cache is being invalidate because project 81eced17da01430395371807a02fd180 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:51.248648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-b49ab58d-bd91-4c9e-8717-86cd91bde874 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:51.249481 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-b49ab58d-bd91-4c9e-8717-86cd91bde874 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:51.250992 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 838/1678] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:50 2022] DELETE /identity/v3/projects/81eced17da01430395371807a02fd180 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:51.252592 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eba7c4-95a2-4536-9f81-b65d78a87ec3 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:51.253117 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eba7c4-95a2-4536-9f81-b65d78a87ec3 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:51.253536 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eba7c4-95a2-4536-9f81-b65d78a87ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:51.283458 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77bab36-3ad6-4cab-b204-7d151df2948b admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:51.285269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-43eba7c4-95a2-4536-9f81-b65d78a87ec3 None None] MFA Rules not processed for user `0a901f6717074380a9c673fcaf2f52b6`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:51.288340 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-43eba7c4-95a2-4536-9f81-b65d78a87ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:51.292797 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442a9e0-fb7c-4439-8748-e437aab62e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:51.293764 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442a9e0-fb7c-4439-8748-e437aab62e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:51.294234 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442a9e0-fb7c-4439-8748-e437aab62e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:51.294693 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e442a9e0-fb7c-4439-8748-e437aab62e50 admin admin] PATH_INFO: `/v3/projects/b9e3d213c9c049bfa91c4fffd5d60f17` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:51.296251 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e442a9e0-fb7c-4439-8748-e437aab62e50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9e3d213c9c049bfa91c4fffd5d60f17)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:51.298428 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e442a9e0-fb7c-4439-8748-e437aab62e50 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:51.369975 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 839/1679] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:51 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:51.370984 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eba7c4-95a2-4536-9f81-b65d78a87ec3 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:51.372545 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-43eba7c4-95a2-4536-9f81-b65d78a87ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:51.389016 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eba7c4-95a2-4536-9f81-b65d78a87ec3 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:51.404485 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-e442a9e0-fb7c-4439-8748-e437aab62e50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9e3d213c9c049bfa91c4fffd5d60f17', 'request_id': 'req-e442a9e0-fb7c-4439-8748-e437aab62e50'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:51.447543 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d33d7-cbaf-4bfb-b797-3e192c7084ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:51.448206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d33d7-cbaf-4bfb-b797-3e192c7084ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:51.448488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d33d7-cbaf-4bfb-b797-3e192c7084ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:51.448761 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d33d7-cbaf-4bfb-b797-3e192c7084ae admin admin] PATH_INFO: `/v3/users/a686fd3aa14243c5963d0b2913beb941` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:51.449892 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39d33d7-cbaf-4bfb-b797-3e192c7084ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=a686fd3aa14243c5963d0b2913beb941)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:51.451837 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39d33d7-cbaf-4bfb-b797-3e192c7084ae admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:51.487869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-e442a9e0-fb7c-4439-8748-e437aab62e50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9e3d213c9c049bfa91c4fffd5d60f17', 'request_id': 'req-e442a9e0-fb7c-4439-8748-e437aab62e50'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:51.493931 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-e442a9e0-fb7c-4439-8748-e437aab62e50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9e3d213c9c049bfa91c4fffd5d60f17', 'request_id': 'req-e442a9e0-fb7c-4439-8748-e437aab62e50'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:51.505487 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-e442a9e0-fb7c-4439-8748-e437aab62e50 admin admin] The token cache is being invalidate because project b9e3d213c9c049bfa91c4fffd5d60f17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:51.505811 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-e442a9e0-fb7c-4439-8748-e437aab62e50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:51.506478 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-e442a9e0-fb7c-4439-8748-e437aab62e50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:51.508004 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 841/1680] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:51 2022] DELETE /identity/v3/projects/b9e3d213c9c049bfa91c4fffd5d60f17 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:51.508776 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442a9e0-fb7c-4439-8748-e437aab62e50 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:51.510049 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-e442a9e0-fb7c-4439-8748-e437aab62e50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:51.526533 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e442a9e0-fb7c-4439-8748-e437aab62e50 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:51.528900 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4cd747-558a-4e29-b4f3-f1547f52673e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:51.529523 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4cd747-558a-4e29-b4f3-f1547f52673e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:51.529830 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4cd747-558a-4e29-b4f3-f1547f52673e admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:51.530105 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4cd747-558a-4e29-b4f3-f1547f52673e admin admin] PATH_INFO: `/v3/projects/26e0a828f23a4c3da908506a798ae736` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:51.531301 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4cd747-558a-4e29-b4f3-f1547f52673e admin admin] RBAC: Authorizing `identity:delete_project(project_id=26e0a828f23a4c3da908506a798ae736)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:51.533205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4cd747-558a-4e29-b4f3-f1547f52673e admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:51.596622 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:51.662072 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-b39d33d7-cbaf-4bfb-b797-3e192c7084ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a686fd3aa14243c5963d0b2913beb941'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:51.662743 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-b39d33d7-cbaf-4bfb-b797-3e192c7084ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a686fd3aa14243c5963d0b2913beb941'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:51.664625 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-0b4cd747-558a-4e29-b4f3-f1547f52673e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26e0a828f23a4c3da908506a798ae736', 'request_id': 'req-0b4cd747-558a-4e29-b4f3-f1547f52673e'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:51.756024 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-b39d33d7-cbaf-4bfb-b797-3e192c7084ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a686fd3aa14243c5963d0b2913beb941'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:51.770641 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-b39d33d7-cbaf-4bfb-b797-3e192c7084ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a686fd3aa14243c5963d0b2913beb941'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:51.771372 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-b39d33d7-cbaf-4bfb-b797-3e192c7084ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a686fd3aa14243c5963d0b2913beb941'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:51.783890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 840/1681] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:51:51 2022] DELETE /identity/v3/users/a686fd3aa14243c5963d0b2913beb941 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:51.784700 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d33d7-cbaf-4bfb-b797-3e192c7084ae admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:51.786049 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-b39d33d7-cbaf-4bfb-b797-3e192c7084ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:51.816315 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-0b4cd747-558a-4e29-b4f3-f1547f52673e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26e0a828f23a4c3da908506a798ae736', 'request_id': 'req-0b4cd747-558a-4e29-b4f3-f1547f52673e'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:51.824129 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-0b4cd747-558a-4e29-b4f3-f1547f52673e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26e0a828f23a4c3da908506a798ae736', 'request_id': 'req-0b4cd747-558a-4e29-b4f3-f1547f52673e'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:51.838607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-0b4cd747-558a-4e29-b4f3-f1547f52673e admin admin] The token cache is being invalidate because project 26e0a828f23a4c3da908506a798ae736 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:51.838902 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-0b4cd747-558a-4e29-b4f3-f1547f52673e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:51.839563 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-0b4cd747-558a-4e29-b4f3-f1547f52673e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:51.845943 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 842/1682] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:51 2022] DELETE /identity/v3/projects/26e0a828f23a4c3da908506a798ae736 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:51.847304 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4cd747-558a-4e29-b4f3-f1547f52673e admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:51.848683 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39d33d7-cbaf-4bfb-b797-3e192c7084ae admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:51.848870 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-0b4cd747-558a-4e29-b4f3-f1547f52673e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:51.853524 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8855cf9c-0e5f-427b-9565-f8a98f431967 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:51.854177 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855cf9c-0e5f-427b-9565-f8a98f431967 service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:51.854466 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855cf9c-0e5f-427b-9565-f8a98f431967 service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:51.854770 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855cf9c-0e5f-427b-9565-f8a98f431967 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:51.856575 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-8855cf9c-0e5f-427b-9565-f8a98f431967 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:51.921888 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8855cf9c-0e5f-427b-9565-f8a98f431967 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:51.923326 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8855cf9c-0e5f-427b-9565-f8a98f431967 service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:51.924100 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4cd747-558a-4e29-b4f3-f1547f52673e admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:51.932411 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 841/1683] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:51 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:51.933667 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d857e66-ffe2-46d4-9189-a5bd5ee9ee20 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:51.933989 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d857e66-ffe2-46d4-9189-a5bd5ee9ee20 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:51.934260 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d857e66-ffe2-46d4-9189-a5bd5ee9ee20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:51.958591 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-3d857e66-ffe2-46d4-9189-a5bd5ee9ee20 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:51.959846 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-3d857e66-ffe2-46d4-9189-a5bd5ee9ee20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:51.983118 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66d010-0c0a-4ff3-b573-0e325fcbf606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:51.984110 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66d010-0c0a-4ff3-b573-0e325fcbf606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:51.984110 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66d010-0c0a-4ff3-b573-0e325fcbf606 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:51.984363 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66d010-0c0a-4ff3-b573-0e325fcbf606 admin admin] PATH_INFO: `/v3/users/a66ee61c4d5b47209a4f1eb90809f38e` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:51.985502 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66d010-0c0a-4ff3-b573-0e325fcbf606 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a66ee61c4d5b47209a4f1eb90809f38e)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:51.987402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de66d010-0c0a-4ff3-b573-0e325fcbf606 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:52.005969 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:52.024569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 842/1684] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:52.025584 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d857e66-ffe2-46d4-9189-a5bd5ee9ee20 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:52.031683 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-3d857e66-ffe2-46d4-9189-a5bd5ee9ee20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:52.044165 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d857e66-ffe2-46d4-9189-a5bd5ee9ee20 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:52.051420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d321c59-7aa4-4dab-8333-0cb4c95aa0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:52.052008 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d321c59-7aa4-4dab-8333-0cb4c95aa0b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:52.052283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d321c59-7aa4-4dab-8333-0cb4c95aa0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:52.052577 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d321c59-7aa4-4dab-8333-0cb4c95aa0b8 admin admin] PATH_INFO: `/v3/users/0a901f6717074380a9c673fcaf2f52b6` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:52.053742 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d321c59-7aa4-4dab-8333-0cb4c95aa0b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a901f6717074380a9c673fcaf2f52b6)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:52.056153 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d321c59-7aa4-4dab-8333-0cb4c95aa0b8 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:52.171627 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-de66d010-0c0a-4ff3-b573-0e325fcbf606 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a66ee61c4d5b47209a4f1eb90809f38e'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:52.310951 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-de66d010-0c0a-4ff3-b573-0e325fcbf606 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a66ee61c4d5b47209a4f1eb90809f38e'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:52.326034 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-de66d010-0c0a-4ff3-b573-0e325fcbf606 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a66ee61c4d5b47209a4f1eb90809f38e'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:52.326910 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-de66d010-0c0a-4ff3-b573-0e325fcbf606 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a66ee61c4d5b47209a4f1eb90809f38e'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:52.327552 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-de66d010-0c0a-4ff3-b573-0e325fcbf606 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a66ee61c4d5b47209a4f1eb90809f38e'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:52.329830 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3d321c59-7aa4-4dab-8333-0cb4c95aa0b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a901f6717074380a9c673fcaf2f52b6'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:52.331118 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3d321c59-7aa4-4dab-8333-0cb4c95aa0b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a901f6717074380a9c673fcaf2f52b6'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:52.342114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 843/1685] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:51:51 2022] DELETE /identity/v3/users/a66ee61c4d5b47209a4f1eb90809f38e => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:52.345175 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63940b46-a67e-43c9-b43a-d432619eb60e None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:52.345175 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63940b46-a67e-43c9-b43a-d432619eb60e None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:52.345175 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63940b46-a67e-43c9-b43a-d432619eb60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:52.366912 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-63940b46-a67e-43c9-b43a-d432619eb60e None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:52.368367 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-63940b46-a67e-43c9-b43a-d432619eb60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:52.387606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:52.413136 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3d321c59-7aa4-4dab-8333-0cb4c95aa0b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a901f6717074380a9c673fcaf2f52b6'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:52.435384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3d321c59-7aa4-4dab-8333-0cb4c95aa0b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a901f6717074380a9c673fcaf2f52b6'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:52.436106 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3d321c59-7aa4-4dab-8333-0cb4c95aa0b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a901f6717074380a9c673fcaf2f52b6'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:52.437219 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 844/1686] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:52.441369 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63940b46-a67e-43c9-b43a-d432619eb60e None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:52.441369 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-63940b46-a67e-43c9-b43a-d432619eb60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:52.452960 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63940b46-a67e-43c9-b43a-d432619eb60e None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:52.462714 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 843/1687] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:51:52 2022] DELETE /identity/v3/users/0a901f6717074380a9c673fcaf2f52b6 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:52.465688 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d321c59-7aa4-4dab-8333-0cb4c95aa0b8 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:52.466841 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819161ce-33d0-48be-bc57-0be5c487f3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:52.467491 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819161ce-33d0-48be-bc57-0be5c487f3f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:52.467800 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819161ce-33d0-48be-bc57-0be5c487f3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:52.468547 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819161ce-33d0-48be-bc57-0be5c487f3f7 admin admin] PATH_INFO: `/v3/projects/e1ce34cf5ce14d34a5d9ef74792615a3` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:52.469402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819161ce-33d0-48be-bc57-0be5c487f3f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1ce34cf5ce14d34a5d9ef74792615a3)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:52.471427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819161ce-33d0-48be-bc57-0be5c487f3f7 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:52.537839 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d321c59-7aa4-4dab-8333-0cb4c95aa0b8 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:52.542617 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3772b6bc-4502-4ef7-9d21-14c0226ead4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:52.543248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772b6bc-4502-4ef7-9d21-14c0226ead4d admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:52.543530 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772b6bc-4502-4ef7-9d21-14c0226ead4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:52.543850 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3772b6bc-4502-4ef7-9d21-14c0226ead4d admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:52.544500 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3772b6bc-4502-4ef7-9d21-14c0226ead4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:51:52.544805 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3772b6bc-4502-4ef7-9d21-14c0226ead4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:52.546398 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3772b6bc-4502-4ef7-9d21-14c0226ead4d admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:52.558561 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 844/1688] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:51:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:51:52.559862 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39309a3-e988-4a64-9f25-56a0a3020495 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:52.560171 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39309a3-e988-4a64-9f25-56a0a3020495 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:52.560456 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39309a3-e988-4a64-9f25-56a0a3020495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:52.579347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-e39309a3-e988-4a64-9f25-56a0a3020495 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:52.580769 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-e39309a3-e988-4a64-9f25-56a0a3020495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:52.584843 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:52.593180 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 845/1689] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:52.594208 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39309a3-e988-4a64-9f25-56a0a3020495 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:52.607760 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39309a3-e988-4a64-9f25-56a0a3020495 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:52.618527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-819161ce-33d0-48be-bc57-0be5c487f3f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1ce34cf5ce14d34a5d9ef74792615a3', 'request_id': 'req-819161ce-33d0-48be-bc57-0be5c487f3f7'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:52.670193 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca845fee-8232-4d4a-8a14-124d25ac74eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:52.670809 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca845fee-8232-4d4a-8a14-124d25ac74eb admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:52.671091 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca845fee-8232-4d4a-8a14-124d25ac74eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:52.671364 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca845fee-8232-4d4a-8a14-124d25ac74eb admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:52.672029 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca845fee-8232-4d4a-8a14-124d25ac74eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:52.673929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca845fee-8232-4d4a-8a14-124d25ac74eb admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:52.674548 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-819161ce-33d0-48be-bc57-0be5c487f3f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1ce34cf5ce14d34a5d9ef74792615a3', 'request_id': 'req-819161ce-33d0-48be-bc57-0be5c487f3f7'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:52.691887 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-819161ce-33d0-48be-bc57-0be5c487f3f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1ce34cf5ce14d34a5d9ef74792615a3', 'request_id': 'req-819161ce-33d0-48be-bc57-0be5c487f3f7'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:52.705156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-819161ce-33d0-48be-bc57-0be5c487f3f7 admin admin] The token cache is being invalidate because project e1ce34cf5ce14d34a5d9ef74792615a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:52.705711 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-819161ce-33d0-48be-bc57-0be5c487f3f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:52.706392 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-819161ce-33d0-48be-bc57-0be5c487f3f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:52.707975 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 845/1690] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:52 2022] DELETE /identity/v3/projects/e1ce34cf5ce14d34a5d9ef74792615a3 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:52.709183 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819161ce-33d0-48be-bc57-0be5c487f3f7 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:52.710517 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-819161ce-33d0-48be-bc57-0be5c487f3f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:52.743712 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:52.759634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 846/1691] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:52 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:52.759634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9a84f-eeea-431a-879e-40f2cf930985 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:52.759634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9a84f-eeea-431a-879e-40f2cf930985 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:52.759634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9a84f-eeea-431a-879e-40f2cf930985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:52.771931 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819161ce-33d0-48be-bc57-0be5c487f3f7 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:52.778395 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-10d9a84f-eeea-431a-879e-40f2cf930985 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:52.779605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-10d9a84f-eeea-431a-879e-40f2cf930985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:52.819067 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96e7b7-5f3d-4e78-a825-9b96b3b16280 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:52.819682 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96e7b7-5f3d-4e78-a825-9b96b3b16280 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:52.819994 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96e7b7-5f3d-4e78-a825-9b96b3b16280 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:52.820268 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96e7b7-5f3d-4e78-a825-9b96b3b16280 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:52.822031 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-bf96e7b7-5f3d-4e78-a825-9b96b3b16280 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:52.843520 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 847/1692] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:52.844985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9a84f-eeea-431a-879e-40f2cf930985 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:52.846141 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-10d9a84f-eeea-431a-879e-40f2cf930985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:52.878357 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9a84f-eeea-431a-879e-40f2cf930985 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:52.878357 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d595b-e7e1-4a31-a405-1804b22838eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:52.878357 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d595b-e7e1-4a31-a405-1804b22838eb admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:52.878357 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d595b-e7e1-4a31-a405-1804b22838eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:52.881603 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666d595b-e7e1-4a31-a405-1804b22838eb admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:52.881603 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666d595b-e7e1-4a31-a405-1804b22838eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:52.881603 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666d595b-e7e1-4a31-a405-1804b22838eb admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:52.899722 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96e7b7-5f3d-4e78-a825-9b96b3b16280 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:52.901175 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96e7b7-5f3d-4e78-a825-9b96b3b16280 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:52.911281 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 846/1693] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:52.912521 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a65e3-4269-427c-b984-aa2dea5f8e41 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:52.912809 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a65e3-4269-427c-b984-aa2dea5f8e41 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:52.913144 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a65e3-4269-427c-b984-aa2dea5f8e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:52.932632 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-4a7a65e3-4269-427c-b984-aa2dea5f8e41 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:52.933837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-4a7a65e3-4269-427c-b984-aa2dea5f8e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:52.941938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 847/1694] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:52.942991 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a65e3-4269-427c-b984-aa2dea5f8e41 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:52.946292 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-4a7a65e3-4269-427c-b984-aa2dea5f8e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:52.961531 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a65e3-4269-427c-b984-aa2dea5f8e41 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:52.963125 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 848/1695] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:52 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:52.966954 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d595b-e7e1-4a31-a405-1804b22838eb admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:52.967175 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc83f2b-1a28-4786-bf6c-017d90e4ae53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:52.968031 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc83f2b-1a28-4786-bf6c-017d90e4ae53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:52.968645 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc83f2b-1a28-4786-bf6c-017d90e4ae53 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:52.968756 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc83f2b-1a28-4786-bf6c-017d90e4ae53 admin admin] PATH_INFO: `/v3/projects/e04fa65e110a435892939347d958e58f` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:52.970052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc83f2b-1a28-4786-bf6c-017d90e4ae53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e04fa65e110a435892939347d958e58f)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:52.972739 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc83f2b-1a28-4786-bf6c-017d90e4ae53 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:52.977571 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666d595b-e7e1-4a31-a405-1804b22838eb admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:52.981681 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:52.987602 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a9abe-8af6-49ad-a1cd-a1a58dfee48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:52.989928 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a9abe-8af6-49ad-a1cd-a1a58dfee48d admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:52.989928 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a9abe-8af6-49ad-a1cd-a1a58dfee48d admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:52.989928 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a9abe-8af6-49ad-a1cd-a1a58dfee48d admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:52.989928 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a9abe-8af6-49ad-a1cd-a1a58dfee48d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:51:52.989928 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a9abe-8af6-49ad-a1cd-a1a58dfee48d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:52.991382 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124a9abe-8af6-49ad-a1cd-a1a58dfee48d admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:53.004084 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 849/1696] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:51:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:51:53.004922 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a9abe-8af6-49ad-a1cd-a1a58dfee48d admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:53.014731 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a9abe-8af6-49ad-a1cd-a1a58dfee48d admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:53.020094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f9fa3-e6ae-4ffc-9038-927c3ce1c8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:53.020687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f9fa3-e6ae-4ffc-9038-927c3ce1c8b8 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:53.020967 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f9fa3-e6ae-4ffc-9038-927c3ce1c8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:53.021238 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f9fa3-e6ae-4ffc-9038-927c3ce1c8b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:53.022526 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f9fa3-e6ae-4ffc-9038-927c3ce1c8b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:53.024213 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f9fa3-e6ae-4ffc-9038-927c3ce1c8b8 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:53.034191 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 850/1697] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:53.035508 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f9fa3-e6ae-4ffc-9038-927c3ce1c8b8 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:53.044972 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f9fa3-e6ae-4ffc-9038-927c3ce1c8b8 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:53.049456 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524991b0-44d5-4d5d-b811-1451ff370f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:53.053104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524991b0-44d5-4d5d-b811-1451ff370f2d admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:53.053104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524991b0-44d5-4d5d-b811-1451ff370f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:53.053104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524991b0-44d5-4d5d-b811-1451ff370f2d admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:53.053104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524991b0-44d5-4d5d-b811-1451ff370f2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:53.053104 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524991b0-44d5-4d5d-b811-1451ff370f2d admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:53.087600 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-4bc83f2b-1a28-4786-bf6c-017d90e4ae53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e04fa65e110a435892939347d958e58f', 'request_id': 'req-4bc83f2b-1a28-4786-bf6c-017d90e4ae53'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:53.158901 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 851/1698] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:53 2022] POST /identity/v3/projects => generated 369 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:53.163415 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524991b0-44d5-4d5d-b811-1451ff370f2d admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:53.179506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-4bc83f2b-1a28-4786-bf6c-017d90e4ae53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e04fa65e110a435892939347d958e58f', 'request_id': 'req-4bc83f2b-1a28-4786-bf6c-017d90e4ae53'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:53.186810 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-4bc83f2b-1a28-4786-bf6c-017d90e4ae53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e04fa65e110a435892939347d958e58f', 'request_id': 'req-4bc83f2b-1a28-4786-bf6c-017d90e4ae53'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:53.199768 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-4bc83f2b-1a28-4786-bf6c-017d90e4ae53 admin admin] The token cache is being invalidate because project e04fa65e110a435892939347d958e58f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:53.200066 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-4bc83f2b-1a28-4786-bf6c-017d90e4ae53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:53.200734 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-4bc83f2b-1a28-4786-bf6c-017d90e4ae53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:53.202248 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 848/1699] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:52 2022] DELETE /identity/v3/projects/e04fa65e110a435892939347d958e58f => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:53.203101 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc83f2b-1a28-4786-bf6c-017d90e4ae53 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:53.204649 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-4bc83f2b-1a28-4786-bf6c-017d90e4ae53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:53.240098 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524991b0-44d5-4d5d-b811-1451ff370f2d admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:53.246648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0bc246-295a-406a-95b4-b19895a5a12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:53.247242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0bc246-295a-406a-95b4-b19895a5a12c admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:53.247661 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0bc246-295a-406a-95b4-b19895a5a12c admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:53.247977 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0bc246-295a-406a-95b4-b19895a5a12c admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:53.252105 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0bc246-295a-406a-95b4-b19895a5a12c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:53.252105 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0bc246-295a-406a-95b4-b19895a5a12c admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:53.262986 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 852/1700] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:53.265216 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a739549-0dcd-48e4-b2bc-fbea88ddca16 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:53.267372 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a739549-0dcd-48e4-b2bc-fbea88ddca16 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:53.267750 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a739549-0dcd-48e4-b2bc-fbea88ddca16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:53.289903 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-5a739549-0dcd-48e4-b2bc-fbea88ddca16 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:53.291084 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc83f2b-1a28-4786-bf6c-017d90e4ae53 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:53.291276 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-5a739549-0dcd-48e4-b2bc-fbea88ddca16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:53.296068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ab69d-fc76-4192-b8c1-805729a28164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:53.296789 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ab69d-fc76-4192-b8c1-805729a28164 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:53.297073 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ab69d-fc76-4192-b8c1-805729a28164 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:53.299282 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ab69d-fc76-4192-b8c1-805729a28164 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:53.299282 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ab69d-fc76-4192-b8c1-805729a28164 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:53.300144 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 853/1701] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:53.300440 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ab69d-fc76-4192-b8c1-805729a28164 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:53.301134 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a739549-0dcd-48e4-b2bc-fbea88ddca16 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:53.302583 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-5a739549-0dcd-48e4-b2bc-fbea88ddca16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:53.315037 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a739549-0dcd-48e4-b2bc-fbea88ddca16 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:53.320270 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700c564-4de0-4d84-b228-cd23ba03c358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:53.320922 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700c564-4de0-4d84-b228-cd23ba03c358 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:53.321201 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700c564-4de0-4d84-b228-cd23ba03c358 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:53.321480 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4700c564-4de0-4d84-b228-cd23ba03c358 admin admin] PATH_INFO: `/v3/projects/78fa502eb85d470b9e9cc4bba1b038d2/users/7f8a269c8ebd400282d97ba8d687ddbd/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:53.336779 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4700c564-4de0-4d84-b228-cd23ba03c358 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78fa502eb85d470b9e9cc4bba1b038d2, user_id=7f8a269c8ebd400282d97ba8d687ddbd, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:53.339439 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4700c564-4de0-4d84-b228-cd23ba03c358 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:53.401960 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 849/1702] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:53 2022] POST /identity/v3/users => generated 285 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:53.403951 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0aeea-e590-41db-a792-927cdaf3e0d9 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:53.404341 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0aeea-e590-41db-a792-927cdaf3e0d9 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:53.404674 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0aeea-e590-41db-a792-927cdaf3e0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:53.427601 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-2db0aeea-e590-41db-a792-927cdaf3e0d9 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:53.428844 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-2db0aeea-e590-41db-a792-927cdaf3e0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:53.436863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 850/1703] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:53.439859 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0aeea-e590-41db-a792-927cdaf3e0d9 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:53.439859 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-2db0aeea-e590-41db-a792-927cdaf3e0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:53.450618 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0aeea-e590-41db-a792-927cdaf3e0d9 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:53.451615 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 854/1704] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:53 2022] PUT /identity/v3/projects/78fa502eb85d470b9e9cc4bba1b038d2/users/7f8a269c8ebd400282d97ba8d687ddbd/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:53.453064 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700c564-4de0-4d84-b228-cd23ba03c358 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:53.510727 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177707af-70d7-4017-a5f3-d43d6a574ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:53.511472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177707af-70d7-4017-a5f3-d43d6a574ed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:53.511835 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177707af-70d7-4017-a5f3-d43d6a574ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:53.512107 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177707af-70d7-4017-a5f3-d43d6a574ed7 admin admin] PATH_INFO: `/v3/projects/ec18b195725f4467bcd946fe679f1ad0` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:53.513256 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177707af-70d7-4017-a5f3-d43d6a574ed7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec18b195725f4467bcd946fe679f1ad0)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:53.515078 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4700c564-4de0-4d84-b228-cd23ba03c358 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:53.515212 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177707af-70d7-4017-a5f3-d43d6a574ed7 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:53.519684 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ac47a-b9c6-4122-8e0b-fe458ef2e16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:53.520611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ac47a-b9c6-4122-8e0b-fe458ef2e16c admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:53.520892 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ac47a-b9c6-4122-8e0b-fe458ef2e16c admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:53.521168 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ac47a-b9c6-4122-8e0b-fe458ef2e16c admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:53.521814 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ac47a-b9c6-4122-8e0b-fe458ef2e16c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:53.523420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62ac47a-b9c6-4122-8e0b-fe458ef2e16c admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:53.525135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:53.533349 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 855/1705] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:53.534154 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ac47a-b9c6-4122-8e0b-fe458ef2e16c admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:53.543609 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ac47a-b9c6-4122-8e0b-fe458ef2e16c admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:53.548244 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f5f57-da23-477f-8684-c3315d3cb50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:53.548879 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f5f57-da23-477f-8684-c3315d3cb50a admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:53.549156 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f5f57-da23-477f-8684-c3315d3cb50a admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:53.549429 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f5f57-da23-477f-8684-c3315d3cb50a admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:53.550574 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f5f57-da23-477f-8684-c3315d3cb50a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:51:53.550877 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f5f57-da23-477f-8684-c3315d3cb50a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:53.552637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f5f57-da23-477f-8684-c3315d3cb50a admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:53.564547 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 856/1706] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:51:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:51:53.566373 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f519d-685a-4a43-8bb1-96f960120877 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:53.566653 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f519d-685a-4a43-8bb1-96f960120877 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:53.566924 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f519d-685a-4a43-8bb1-96f960120877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:53.594395 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-bb2f519d-685a-4a43-8bb1-96f960120877 None None] MFA Rules not processed for user `7f8a269c8ebd400282d97ba8d687ddbd`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:53.595584 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-bb2f519d-685a-4a43-8bb1-96f960120877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:53.618891 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-177707af-70d7-4017-a5f3-d43d6a574ed7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec18b195725f4467bcd946fe679f1ad0', 'request_id': 'req-177707af-70d7-4017-a5f3-d43d6a574ed7'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:53.657609 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 857/1707] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:53.658382 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f519d-685a-4a43-8bb1-96f960120877 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:53.669203 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f519d-685a-4a43-8bb1-96f960120877 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:53.692932 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-177707af-70d7-4017-a5f3-d43d6a574ed7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec18b195725f4467bcd946fe679f1ad0', 'request_id': 'req-177707af-70d7-4017-a5f3-d43d6a574ed7'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:53.699559 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-177707af-70d7-4017-a5f3-d43d6a574ed7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec18b195725f4467bcd946fe679f1ad0', 'request_id': 'req-177707af-70d7-4017-a5f3-d43d6a574ed7'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:53.712883 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-177707af-70d7-4017-a5f3-d43d6a574ed7 admin admin] The token cache is being invalidate because project ec18b195725f4467bcd946fe679f1ad0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:53.713164 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-177707af-70d7-4017-a5f3-d43d6a574ed7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:53.713834 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-177707af-70d7-4017-a5f3-d43d6a574ed7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:53.715357 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 851/1708] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:53 2022] DELETE /identity/v3/projects/ec18b195725f4467bcd946fe679f1ad0 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:53.716145 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177707af-70d7-4017-a5f3-d43d6a574ed7 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:53.717406 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-177707af-70d7-4017-a5f3-d43d6a574ed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:53.729269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254dde37-45ee-4567-87e4-7383e2368dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:53.729888 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254dde37-45ee-4567-87e4-7383e2368dba admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:53.730169 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254dde37-45ee-4567-87e4-7383e2368dba admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:53.730444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254dde37-45ee-4567-87e4-7383e2368dba admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:53.731088 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254dde37-45ee-4567-87e4-7383e2368dba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:53.732708 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254dde37-45ee-4567-87e4-7383e2368dba admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:53.746273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 858/1709] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:53.746273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254dde37-45ee-4567-87e4-7383e2368dba admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:53.746273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-254dde37-45ee-4567-87e4-7383e2368dba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:53.783785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177707af-70d7-4017-a5f3-d43d6a574ed7 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:53.788612 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8515c60-cfdd-4562-b004-e680163b0fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:53.789219 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8515c60-cfdd-4562-b004-e680163b0fa4 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:53.789500 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8515c60-cfdd-4562-b004-e680163b0fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:53.789799 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8515c60-cfdd-4562-b004-e680163b0fa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:53.790438 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8515c60-cfdd-4562-b004-e680163b0fa4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:53.792415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8515c60-cfdd-4562-b004-e680163b0fa4 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:53.802581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254dde37-45ee-4567-87e4-7383e2368dba admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:53.806281 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa08644-5198-457d-add6-42e0dfedb7fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:53.806887 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa08644-5198-457d-add6-42e0dfedb7fe service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:53.807166 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa08644-5198-457d-add6-42e0dfedb7fe service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:53.807440 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa08644-5198-457d-add6-42e0dfedb7fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:53.809264 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-3aa08644-5198-457d-add6-42e0dfedb7fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:53.867272 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa08644-5198-457d-add6-42e0dfedb7fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:53.868730 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa08644-5198-457d-add6-42e0dfedb7fe service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:53.877670 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 859/1710] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:53.878870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922b57b-f7d4-4477-9724-8ca767c4f950 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:53.879159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922b57b-f7d4-4477-9724-8ca767c4f950 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:53.879440 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5922b57b-f7d4-4477-9724-8ca767c4f950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:53.888996 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 852/1711] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:53 2022] POST /identity/v3/projects => generated 401 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:53.889773 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8515c60-cfdd-4562-b004-e680163b0fa4 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:53.893161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c8515c60-cfdd-4562-b004-e680163b0fa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:53.913819 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-5922b57b-f7d4-4477-9724-8ca767c4f950 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:53.916386 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-5922b57b-f7d4-4477-9724-8ca767c4f950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:53.957395 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:53.966159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8515c60-cfdd-4562-b004-e680163b0fa4 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:53.970780 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e780674f-85ca-464a-bd80-0d333e951272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:53.971443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780674f-85ca-464a-bd80-0d333e951272 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:53.972267 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780674f-85ca-464a-bd80-0d333e951272 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:53.972548 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780674f-85ca-464a-bd80-0d333e951272 admin admin] PATH_INFO: `/v3/projects/226cbf5a2fc74c2f96023776b52f85a5/users/fc753dbadafa493db7b2c2b4b5ad4289/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:53.985911 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 860/1712] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:53.986872 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922b57b-f7d4-4477-9724-8ca767c4f950 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:53.993884 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e780674f-85ca-464a-bd80-0d333e951272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=226cbf5a2fc74c2f96023776b52f85a5, user_id=fc753dbadafa493db7b2c2b4b5ad4289, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:53.996618 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5922b57b-f7d4-4477-9724-8ca767c4f950 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:53.996759 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e780674f-85ca-464a-bd80-0d333e951272 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:54.001186 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd11d38d-4b89-433e-8d04-9f289d6b0ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:54.001778 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11d38d-4b89-433e-8d04-9f289d6b0ecd admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:54.002056 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11d38d-4b89-433e-8d04-9f289d6b0ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:54.002326 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd11d38d-4b89-433e-8d04-9f289d6b0ecd admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:54.002979 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11d38d-4b89-433e-8d04-9f289d6b0ecd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:54.005065 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd11d38d-4b89-433e-8d04-9f289d6b0ecd admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:54.092835 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 853/1713] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:53 2022] PUT /identity/v3/projects/226cbf5a2fc74c2f96023776b52f85a5/users/fc753dbadafa493db7b2c2b4b5ad4289/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:54.093710 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e780674f-85ca-464a-bd80-0d333e951272 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:54.095300 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-e780674f-85ca-464a-bd80-0d333e951272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:54.120687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 861/1714] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:53 2022] POST /identity/v3/users => generated 301 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:54.120687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af396d-1f8b-48ab-a981-ca714e1bd083 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:54.120687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af396d-1f8b-48ab-a981-ca714e1bd083 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:54.120687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af396d-1f8b-48ab-a981-ca714e1bd083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:54.139337 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-23af396d-1f8b-48ab-a981-ca714e1bd083 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:54.140690 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-23af396d-1f8b-48ab-a981-ca714e1bd083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:54.158139 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e780674f-85ca-464a-bd80-0d333e951272 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:54.163664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c59c6-82df-4f24-b493-523a8ce4abec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:54.164452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c59c6-82df-4f24-b493-523a8ce4abec admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:54.164736 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c59c6-82df-4f24-b493-523a8ce4abec admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:54.164967 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124c59c6-82df-4f24-b493-523a8ce4abec admin admin] PATH_INFO: `/v3/users/7f8a269c8ebd400282d97ba8d687ddbd` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:54.166105 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124c59c6-82df-4f24-b493-523a8ce4abec admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f8a269c8ebd400282d97ba8d687ddbd)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:54.168076 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124c59c6-82df-4f24-b493-523a8ce4abec admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:54.199697 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 862/1715] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:54.200978 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fcdfa9-1f06-4f57-9d70-7ea33ee63e1f None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:54.201267 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fcdfa9-1f06-4f57-9d70-7ea33ee63e1f None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:54.201543 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fcdfa9-1f06-4f57-9d70-7ea33ee63e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:54.219115 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:54.228844 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-60fcdfa9-1f06-4f57-9d70-7ea33ee63e1f None None] MFA Rules not processed for user `fc753dbadafa493db7b2c2b4b5ad4289`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:54.230054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-60fcdfa9-1f06-4f57-9d70-7ea33ee63e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:54.293134 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 863/1716] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:54.293971 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fcdfa9-1f06-4f57-9d70-7ea33ee63e1f None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:54.304056 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fcdfa9-1f06-4f57-9d70-7ea33ee63e1f None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:54.308764 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d8227-eb21-4559-873f-56d9af815cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:54.309360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d8227-eb21-4559-873f-56d9af815cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:54.309638 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d8227-eb21-4559-873f-56d9af815cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:54.309909 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d8227-eb21-4559-873f-56d9af815cf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:54.310546 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d8227-eb21-4559-873f-56d9af815cf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:54.312185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d8227-eb21-4559-873f-56d9af815cf6 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:54.322877 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 864/1717] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:54.323778 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d8227-eb21-4559-873f-56d9af815cf6 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:54.325212 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:54.335459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d8227-eb21-4559-873f-56d9af815cf6 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:54.339994 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8138c-cc1a-4745-a2fd-ee7271b8a141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:54.340596 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8138c-cc1a-4745-a2fd-ee7271b8a141 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:54.340822 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8138c-cc1a-4745-a2fd-ee7271b8a141 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:54.341094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8138c-cc1a-4745-a2fd-ee7271b8a141 admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:54.341710 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8138c-cc1a-4745-a2fd-ee7271b8a141 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:51:54.342011 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8138c-cc1a-4745-a2fd-ee7271b8a141 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:54.343657 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8138c-cc1a-4745-a2fd-ee7271b8a141 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:54.356930 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 865/1718] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:51:54.357695 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8138c-cc1a-4745-a2fd-ee7271b8a141 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:54.359249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-28f8138c-cc1a-4745-a2fd-ee7271b8a141 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:54.393151 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-124c59c6-82df-4f24-b493-523a8ce4abec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f8a269c8ebd400282d97ba8d687ddbd'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:54.435498 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8138c-cc1a-4745-a2fd-ee7271b8a141 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:54.439202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27ec413-5aa1-49db-b013-0fcdff25f6d2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '565a836c7f89478cad615bd6c14531a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:54.440084 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ec413-5aa1-49db-b013-0fcdff25f6d2 service glance] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:54.440542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ec413-5aa1-49db-b013-0fcdff25f6d2 service glance] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:54.441496 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ec413-5aa1-49db-b013-0fcdff25f6d2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:54.450177 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27ec413-5aa1-49db-b013-0fcdff25f6d2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:54.451919 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27ec413-5aa1-49db-b013-0fcdff25f6d2 service glance] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:54.461635 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 866/1719] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:54 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:54.462352 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27ec413-5aa1-49db-b013-0fcdff25f6d2 service glance] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:54.464950 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-124c59c6-82df-4f24-b493-523a8ce4abec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f8a269c8ebd400282d97ba8d687ddbd'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:54.474876 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27ec413-5aa1-49db-b013-0fcdff25f6d2 service glance] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:54.480032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-124c59c6-82df-4f24-b493-523a8ce4abec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f8a269c8ebd400282d97ba8d687ddbd'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:54.480316 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba507a9-8fe4-4ffa-aee3-73082d148dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:54.481039 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba507a9-8fe4-4ffa-aee3-73082d148dd0 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:54.481149 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-124c59c6-82df-4f24-b493-523a8ce4abec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f8a269c8ebd400282d97ba8d687ddbd'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:54.486318 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba507a9-8fe4-4ffa-aee3-73082d148dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:54.486318 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba507a9-8fe4-4ffa-aee3-73082d148dd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:54.486509 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-124c59c6-82df-4f24-b493-523a8ce4abec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f8a269c8ebd400282d97ba8d687ddbd'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:54.486599 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba507a9-8fe4-4ffa-aee3-73082d148dd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:54.486599 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba507a9-8fe4-4ffa-aee3-73082d148dd0 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:54.502929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 867/1720] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:54.502929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba507a9-8fe4-4ffa-aee3-73082d148dd0 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:54.504414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-dba507a9-8fe4-4ffa-aee3-73082d148dd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:54.514542 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 854/1721] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:51:54 2022] DELETE /identity/v3/users/7f8a269c8ebd400282d97ba8d687ddbd => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:54.515399 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c59c6-82df-4f24-b493-523a8ce4abec admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:54.519969 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-124c59c6-82df-4f24-b493-523a8ce4abec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:54.525502 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba507a9-8fe4-4ffa-aee3-73082d148dd0 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:54.536068 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:54.567726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:54.609431 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda712d-e2c1-4730-b13f-7079c7eeaf58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:54.610079 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda712d-e2c1-4730-b13f-7079c7eeaf58 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:54.610559 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda712d-e2c1-4730-b13f-7079c7eeaf58 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:54.610767 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda712d-e2c1-4730-b13f-7079c7eeaf58 admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:54.611140 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124c59c6-82df-4f24-b493-523a8ce4abec admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:54.611657 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda712d-e2c1-4730-b13f-7079c7eeaf58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:54.613773 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda712d-e2c1-4730-b13f-7079c7eeaf58 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:54.622857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d28b7c-9c2d-494f-8247-3695754a040e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:54.623910 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d28b7c-9c2d-494f-8247-3695754a040e admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:54.627293 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d28b7c-9c2d-494f-8247-3695754a040e admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:54.627293 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d28b7c-9c2d-494f-8247-3695754a040e admin admin] PATH_INFO: `/v3/projects/5828f1a4f3e04eb0acde9539b1acf0e5/users/4111677cd96f4984aaae95f11fbd82cb/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:54.641349 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d28b7c-9c2d-494f-8247-3695754a040e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5828f1a4f3e04eb0acde9539b1acf0e5, user_id=4111677cd96f4984aaae95f11fbd82cb, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:54.644704 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d28b7c-9c2d-494f-8247-3695754a040e admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:54.649070 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:54.707640 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 868/1722] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:54 2022] POST /identity/v3/projects => generated 373 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:54.709018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48daa879-b33d-4ba4-a6c8-78e9ef8f1b7d None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:54.709306 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48daa879-b33d-4ba4-a6c8-78e9ef8f1b7d None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:54.709570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48daa879-b33d-4ba4-a6c8-78e9ef8f1b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:54.726712 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 855/1723] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:54 2022] PUT /identity/v3/projects/5828f1a4f3e04eb0acde9539b1acf0e5/users/4111677cd96f4984aaae95f11fbd82cb/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:54.727922 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d28b7c-9c2d-494f-8247-3695754a040e admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:54.728302 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-48daa879-b33d-4ba4-a6c8-78e9ef8f1b7d None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:54.729499 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-48daa879-b33d-4ba4-a6c8-78e9ef8f1b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:54.804929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 869/1724] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:54.805875 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d28b7c-9c2d-494f-8247-3695754a040e admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:54.806564 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46e104-0a13-4582-9f3b-d96a1a125e2a None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:54.806921 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46e104-0a13-4582-9f3b-d96a1a125e2a None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:54.807124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46e104-0a13-4582-9f3b-d96a1a125e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:54.810897 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab65b3-3242-40a7-97b9-6e57fae59db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:54.811520 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab65b3-3242-40a7-97b9-6e57fae59db4 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:54.811879 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab65b3-3242-40a7-97b9-6e57fae59db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:54.812157 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab65b3-3242-40a7-97b9-6e57fae59db4 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:54.812793 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab65b3-3242-40a7-97b9-6e57fae59db4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:54.814894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ab65b3-3242-40a7-97b9-6e57fae59db4 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:54.838262 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-aa46e104-0a13-4582-9f3b-d96a1a125e2a None None] MFA Rules not processed for user `4111677cd96f4984aaae95f11fbd82cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:54.839475 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-aa46e104-0a13-4582-9f3b-d96a1a125e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:54.886297 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:54.893292 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 856/1725] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:54 2022] POST /identity/v3/users => generated 287 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:54.894786 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab65b3-3242-40a7-97b9-6e57fae59db4 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:54.896393 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-f2ab65b3-3242-40a7-97b9-6e57fae59db4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:54.904508 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 870/1726] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:54 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:54.905398 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46e104-0a13-4582-9f3b-d96a1a125e2a None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:54.916429 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46e104-0a13-4582-9f3b-d96a1a125e2a None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:54.922682 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:54.922825 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e186f04-8737-4cac-9f0e-df0c0d237b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:54.923416 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e186f04-8737-4cac-9f0e-df0c0d237b98 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:54.923694 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e186f04-8737-4cac-9f0e-df0c0d237b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:54.924000 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e186f04-8737-4cac-9f0e-df0c0d237b98 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:54.925215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e186f04-8737-4cac-9f0e-df0c0d237b98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:54.927529 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e186f04-8737-4cac-9f0e-df0c0d237b98 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:54.943904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 871/1727] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:54.943904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e186f04-8737-4cac-9f0e-df0c0d237b98 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:54.951129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e186f04-8737-4cac-9f0e-df0c0d237b98 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:54.957236 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab65b3-3242-40a7-97b9-6e57fae59db4 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:54.958225 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed8dbf-0367-41df-ae26-125e5fb4cdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:54.958981 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed8dbf-0367-41df-ae26-125e5fb4cdce admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:54.959576 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed8dbf-0367-41df-ae26-125e5fb4cdce admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:54.960072 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed8dbf-0367-41df-ae26-125e5fb4cdce admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:54.961267 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed8dbf-0367-41df-ae26-125e5fb4cdce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:54.961389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8264e-6600-4196-acaa-1d2f8a27c377 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:54.961907 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8264e-6600-4196-acaa-1d2f8a27c377 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:54.962190 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8264e-6600-4196-acaa-1d2f8a27c377 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:54.962730 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8264e-6600-4196-acaa-1d2f8a27c377 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:54.965091 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed8dbf-0367-41df-ae26-125e5fb4cdce admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:54.972053 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8264e-6600-4196-acaa-1d2f8a27c377 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:54.973500 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d8264e-6600-4196-acaa-1d2f8a27c377 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:54.984005 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 857/1728] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:54.985002 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8264e-6600-4196-acaa-1d2f8a27c377 service neutron] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:54.995270 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8264e-6600-4196-acaa-1d2f8a27c377 service neutron] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:55.000392 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd053d4-d5fa-4904-8cc4-fe98433654f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:55.003946 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd053d4-d5fa-4904-8cc4-fe98433654f7 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:55.003946 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd053d4-d5fa-4904-8cc4-fe98433654f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:55.003946 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd053d4-d5fa-4904-8cc4-fe98433654f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:55.003946 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd053d4-d5fa-4904-8cc4-fe98433654f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:55.005187 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd053d4-d5fa-4904-8cc4-fe98433654f7 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:55.016020 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 858/1729] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:55.020166 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd053d4-d5fa-4904-8cc4-fe98433654f7 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:55.030873 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd053d4-d5fa-4904-8cc4-fe98433654f7 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:55.035757 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29743686-e661-493e-8226-83c03e6f2208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:55.036427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29743686-e661-493e-8226-83c03e6f2208 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:55.036705 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29743686-e661-493e-8226-83c03e6f2208 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:55.037007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29743686-e661-493e-8226-83c03e6f2208 admin admin] PATH_INFO: `/v3/projects/2a1b57f080dc4f648fd96a4c1f66fe24/users/4269a71ef3264e6eae68afc3a931af85/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:55.054242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 872/1730] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:54 2022] POST /identity/v3/projects => generated 401 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:55.054480 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29743686-e661-493e-8226-83c03e6f2208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a1b57f080dc4f648fd96a4c1f66fe24, user_id=4269a71ef3264e6eae68afc3a931af85, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:55.058175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed8dbf-0367-41df-ae26-125e5fb4cdce admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:55.058386 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29743686-e661-493e-8226-83c03e6f2208 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:55.122408 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed8dbf-0367-41df-ae26-125e5fb4cdce admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:55.127199 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50def602-08a7-492a-a090-17ea70125db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:55.127898 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50def602-08a7-492a-a090-17ea70125db9 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:55.128707 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50def602-08a7-492a-a090-17ea70125db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:55.128986 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50def602-08a7-492a-a090-17ea70125db9 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:55.129658 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50def602-08a7-492a-a090-17ea70125db9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:55.131793 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50def602-08a7-492a-a090-17ea70125db9 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:55.157011 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 859/1731] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:55 2022] PUT /identity/v3/projects/2a1b57f080dc4f648fd96a4c1f66fe24/users/4269a71ef3264e6eae68afc3a931af85/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:51:55.168223 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24840f12-34ce-4470-9bbd-37066ae7b353 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:55.168570 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24840f12-34ce-4470-9bbd-37066ae7b353 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:55.168911 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24840f12-34ce-4470-9bbd-37066ae7b353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:55.198308 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-24840f12-34ce-4470-9bbd-37066ae7b353 None None] MFA Rules not processed for user `4269a71ef3264e6eae68afc3a931af85`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:55.199538 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-24840f12-34ce-4470-9bbd-37066ae7b353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:55.216313 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 873/1732] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:55 2022] POST /identity/v3/users => generated 307 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:51:55.220140 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50def602-08a7-492a-a090-17ea70125db9 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:55.231321 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50def602-08a7-492a-a090-17ea70125db9 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:55.269974 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 860/1733] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:51:55.274393 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24840f12-34ce-4470-9bbd-37066ae7b353 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:55.302580 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d7b9f9-5551-4377-8623-7ef4eb1334fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:55.303249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7b9f9-5551-4377-8623-7ef4eb1334fe admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:55.303531 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7b9f9-5551-4377-8623-7ef4eb1334fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:55.303834 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7b9f9-5551-4377-8623-7ef4eb1334fe admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:55.304477 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d7b9f9-5551-4377-8623-7ef4eb1334fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:55.306217 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d7b9f9-5551-4377-8623-7ef4eb1334fe admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:55.317653 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 874/1734] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:51:55.321781 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d7b9f9-5551-4377-8623-7ef4eb1334fe admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:55.334318 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d7b9f9-5551-4377-8623-7ef4eb1334fe admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:55.339432 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68447d0-6345-41b5-afce-3213069ddc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:55.340339 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68447d0-6345-41b5-afce-3213069ddc1e admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:55.340905 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68447d0-6345-41b5-afce-3213069ddc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:55.341092 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68447d0-6345-41b5-afce-3213069ddc1e admin admin] PATH_INFO: `/v3/projects/ed29e0adff8a48feab6eb3b7447d9532/users/17fe64aa8ed04d1bb29f8ce42d898e3e/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:55.349174 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24840f12-34ce-4470-9bbd-37066ae7b353 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:55.354229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee30e770-2bb1-4925-acc4-8031d1087d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:55.354851 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee30e770-2bb1-4925-acc4-8031d1087d98 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:55.355322 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee30e770-2bb1-4925-acc4-8031d1087d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:55.355603 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee30e770-2bb1-4925-acc4-8031d1087d98 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:55.356276 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee30e770-2bb1-4925-acc4-8031d1087d98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:55.358180 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee30e770-2bb1-4925-acc4-8031d1087d98 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:55.363009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68447d0-6345-41b5-afce-3213069ddc1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed29e0adff8a48feab6eb3b7447d9532, user_id=17fe64aa8ed04d1bb29f8ce42d898e3e, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:55.366134 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68447d0-6345-41b5-afce-3213069ddc1e admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:55.465186 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 875/1735] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:55 2022] PUT /identity/v3/projects/ed29e0adff8a48feab6eb3b7447d9532/users/17fe64aa8ed04d1bb29f8ce42d898e3e/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:51:55.466547 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68447d0-6345-41b5-afce-3213069ddc1e admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:55.468642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-f68447d0-6345-41b5-afce-3213069ddc1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:55.470568 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 861/1736] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:55 2022] POST /identity/v3/projects => generated 375 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:51:55.471663 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee30e770-2bb1-4925-acc4-8031d1087d98 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:55.478900 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:55.495261 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:55.547668 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee30e770-2bb1-4925-acc4-8031d1087d98 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:55.549891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68447d0-6345-41b5-afce-3213069ddc1e admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:55.553047 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962e813-c546-4298-995c-b85d830d6ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:55.553684 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962e813-c546-4298-995c-b85d830d6ad0 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:55.554081 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962e813-c546-4298-995c-b85d830d6ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:55.554336 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962e813-c546-4298-995c-b85d830d6ad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:55.554821 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297019a5-4aa5-42e5-938a-0010c9d7f75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:55.554985 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a962e813-c546-4298-995c-b85d830d6ad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:55.555457 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297019a5-4aa5-42e5-938a-0010c9d7f75e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:55.555765 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297019a5-4aa5-42e5-938a-0010c9d7f75e admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:55.556044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297019a5-4aa5-42e5-938a-0010c9d7f75e admin admin] PATH_INFO: `/v3/projects/78fa502eb85d470b9e9cc4bba1b038d2` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:55.556591 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a962e813-c546-4298-995c-b85d830d6ad0 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:55.557167 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297019a5-4aa5-42e5-938a-0010c9d7f75e admin admin] RBAC: Authorizing `identity:delete_project(project_id=78fa502eb85d470b9e9cc4bba1b038d2)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:55.559134 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297019a5-4aa5-42e5-938a-0010c9d7f75e admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:55.570416 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 862/1737] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:55.570416 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962e813-c546-4298-995c-b85d830d6ad0 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:55.579692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962e813-c546-4298-995c-b85d830d6ad0 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:55.585373 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202d1ef-ec28-433c-824f-c8b1243a3ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:55.586071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202d1ef-ec28-433c-824f-c8b1243a3ce2 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:55.586467 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202d1ef-ec28-433c-824f-c8b1243a3ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:55.586812 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202d1ef-ec28-433c-824f-c8b1243a3ce2 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:55.587500 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202d1ef-ec28-433c-824f-c8b1243a3ce2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:55.589471 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202d1ef-ec28-433c-824f-c8b1243a3ce2 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:55.657173 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 863/1738] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:55 2022] POST /identity/v3/users => generated 294 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:55.658140 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202d1ef-ec28-433c-824f-c8b1243a3ce2 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:55.668893 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-297019a5-4aa5-42e5-938a-0010c9d7f75e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78fa502eb85d470b9e9cc4bba1b038d2', 'request_id': 'req-297019a5-4aa5-42e5-938a-0010c9d7f75e'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:55.669062 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202d1ef-ec28-433c-824f-c8b1243a3ce2 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:55.738308 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a8f69-5063-4625-854a-1a6a2cc2f1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:55.738979 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a8f69-5063-4625-854a-1a6a2cc2f1a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:55.739260 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a8f69-5063-4625-854a-1a6a2cc2f1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:55.739538 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a8f69-5063-4625-854a-1a6a2cc2f1a0 admin admin] PATH_INFO: `/v3/projects/ed29e0adff8a48feab6eb3b7447d9532/users/17fe64aa8ed04d1bb29f8ce42d898e3e/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:55.741550 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a8f69-5063-4625-854a-1a6a2cc2f1a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed29e0adff8a48feab6eb3b7447d9532, user_id=17fe64aa8ed04d1bb29f8ce42d898e3e, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:55.744261 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a8f69-5063-4625-854a-1a6a2cc2f1a0 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:55.754920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-297019a5-4aa5-42e5-938a-0010c9d7f75e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78fa502eb85d470b9e9cc4bba1b038d2', 'request_id': 'req-297019a5-4aa5-42e5-938a-0010c9d7f75e'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:55.760134 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 864/1739] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:55 2022] PUT /identity/v3/projects/ed29e0adff8a48feab6eb3b7447d9532/users/17fe64aa8ed04d1bb29f8ce42d898e3e/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:55.760970 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a8f69-5063-4625-854a-1a6a2cc2f1a0 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:55.764055 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-297019a5-4aa5-42e5-938a-0010c9d7f75e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78fa502eb85d470b9e9cc4bba1b038d2', 'request_id': 'req-297019a5-4aa5-42e5-938a-0010c9d7f75e'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:55.782358 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-297019a5-4aa5-42e5-938a-0010c9d7f75e admin admin] The token cache is being invalidate because project 78fa502eb85d470b9e9cc4bba1b038d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:51:55.783096 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-297019a5-4aa5-42e5-938a-0010c9d7f75e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:55.783620 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-297019a5-4aa5-42e5-938a-0010c9d7f75e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:51:55.785484 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 876/1740] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:51:55 2022] DELETE /identity/v3/projects/78fa502eb85d470b9e9cc4bba1b038d2 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:55.790053 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756d774-7e56-4beb-88f1-13aee16462d9 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:55.790366 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756d774-7e56-4beb-88f1-13aee16462d9 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:55.790644 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756d774-7e56-4beb-88f1-13aee16462d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:55.821246 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-9756d774-7e56-4beb-88f1-13aee16462d9 None None] MFA Rules not processed for user `17fe64aa8ed04d1bb29f8ce42d898e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:55.822471 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-9756d774-7e56-4beb-88f1-13aee16462d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:55.832167 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a8f69-5063-4625-854a-1a6a2cc2f1a0 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:55.837082 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764d0f95-f870-48ab-b6ea-7c555fc38ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:55.837760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d0f95-f870-48ab-b6ea-7c555fc38ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:55.838051 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d0f95-f870-48ab-b6ea-7c555fc38ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:55.838328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d0f95-f870-48ab-b6ea-7c555fc38ad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:55.838976 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764d0f95-f870-48ab-b6ea-7c555fc38ad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:55.840599 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764d0f95-f870-48ab-b6ea-7c555fc38ad5 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:55.855307 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 865/1741] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:55.856633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da6db9-ebbd-433d-9b31-4af76684882b None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:55.856925 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da6db9-ebbd-433d-9b31-4af76684882b None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:55.857199 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da6db9-ebbd-433d-9b31-4af76684882b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:55.876396 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-70da6db9-ebbd-433d-9b31-4af76684882b None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:55.877603 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-70da6db9-ebbd-433d-9b31-4af76684882b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:55.885399 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 866/1742] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:55.886168 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da6db9-ebbd-433d-9b31-4af76684882b None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:55.895280 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-70da6db9-ebbd-433d-9b31-4af76684882b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:55.899177 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 877/1743] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:55 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:55.900935 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90bfd6-c3f9-4055-9943-9f944f1580de None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:55.901318 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90bfd6-c3f9-4055-9943-9f944f1580de None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:55.901683 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90bfd6-c3f9-4055-9943-9f944f1580de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:55.911486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da6db9-ebbd-433d-9b31-4af76684882b None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:55.916724 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193fb7a-7640-436c-b86f-19a3522af200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:55.917388 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193fb7a-7640-436c-b86f-19a3522af200 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:55.917669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193fb7a-7640-436c-b86f-19a3522af200 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:55.917942 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193fb7a-7640-436c-b86f-19a3522af200 admin admin] PATH_INFO: `/v3/projects/8e904abd3ae04dc3ac5aa84c69fc944e/users/c96f0800e2854379ba9325c36aed0c5e/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:55.920512 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:55.928232 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-ef90bfd6-c3f9-4055-9943-9f944f1580de None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:55.929609 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-ef90bfd6-c3f9-4055-9943-9f944f1580de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:55.935812 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193fb7a-7640-436c-b86f-19a3522af200 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e904abd3ae04dc3ac5aa84c69fc944e, user_id=c96f0800e2854379ba9325c36aed0c5e, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:55.938075 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 878/1744] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:55.938517 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5193fb7a-7640-436c-b86f-19a3522af200 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:55.942574 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90bfd6-c3f9-4055-9943-9f944f1580de None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:55.943893 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-ef90bfd6-c3f9-4055-9943-9f944f1580de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:55.978183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:56.003834 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90bfd6-c3f9-4055-9943-9f944f1580de None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:56.007288 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 867/1745] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:55 2022] PUT /identity/v3/projects/8e904abd3ae04dc3ac5aa84c69fc944e/users/c96f0800e2854379ba9325c36aed0c5e/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:56.008207 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193fb7a-7640-436c-b86f-19a3522af200 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:56.060420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e95b2e-b2e8-4f4f-9552-5d7e8619a3f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:56.061234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e95b2e-b2e8-4f4f-9552-5d7e8619a3f1 service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:56.061737 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e95b2e-b2e8-4f4f-9552-5d7e8619a3f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:56.063103 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e95b2e-b2e8-4f4f-9552-5d7e8619a3f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:56.085824 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193fb7a-7640-436c-b86f-19a3522af200 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:56.090671 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f7a26-5a33-4712-9d26-2f2b9a25642c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:56.091307 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f7a26-5a33-4712-9d26-2f2b9a25642c admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:56.091760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f7a26-5a33-4712-9d26-2f2b9a25642c admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:56.091926 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f7a26-5a33-4712-9d26-2f2b9a25642c admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:56.092576 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f7a26-5a33-4712-9d26-2f2b9a25642c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:51:56.092881 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f7a26-5a33-4712-9d26-2f2b9a25642c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:56.094464 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f7a26-5a33-4712-9d26-2f2b9a25642c admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:56.109215 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 868/1746] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:51:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:51:56.110329 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f7a26-5a33-4712-9d26-2f2b9a25642c admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:56.121311 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f7a26-5a33-4712-9d26-2f2b9a25642c admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:56.126416 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85314ebb-04a4-42c7-a42a-dfd1f60e08c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:56.127018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85314ebb-04a4-42c7-a42a-dfd1f60e08c4 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:56.127330 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85314ebb-04a4-42c7-a42a-dfd1f60e08c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:56.127650 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85314ebb-04a4-42c7-a42a-dfd1f60e08c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:56.128907 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85314ebb-04a4-42c7-a42a-dfd1f60e08c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:56.130744 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85314ebb-04a4-42c7-a42a-dfd1f60e08c4 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:56.136029 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e95b2e-b2e8-4f4f-9552-5d7e8619a3f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:56.137440 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e95b2e-b2e8-4f4f-9552-5d7e8619a3f1 service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:56.144201 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 869/1747] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:56.144685 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85314ebb-04a4-42c7-a42a-dfd1f60e08c4 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:56.152870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 879/1748] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:55 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:51:56.153635 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e95b2e-b2e8-4f4f-9552-5d7e8619a3f1 service cinder] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:56.159132 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85314ebb-04a4-42c7-a42a-dfd1f60e08c4 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:56.165824 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75681c0a-5ad3-4b6a-bd1c-932b7f56b618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:56.166929 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75681c0a-5ad3-4b6a-bd1c-932b7f56b618 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:56.167066 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e95b2e-b2e8-4f4f-9552-5d7e8619a3f1 service cinder] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:56.167363 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75681c0a-5ad3-4b6a-bd1c-932b7f56b618 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:56.168318 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75681c0a-5ad3-4b6a-bd1c-932b7f56b618 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:56.169149 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75681c0a-5ad3-4b6a-bd1c-932b7f56b618 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:56.171614 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75681c0a-5ad3-4b6a-bd1c-932b7f56b618 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:56.173208 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570d14b-f61a-4499-a955-eba712f3913d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:56.174430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570d14b-f61a-4499-a955-eba712f3913d admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:56.175768 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570d14b-f61a-4499-a955-eba712f3913d admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:56.175768 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570d14b-f61a-4499-a955-eba712f3913d admin admin] PATH_INFO: `/v3/projects/8e904abd3ae04dc3ac5aa84c69fc944e/users/c96f0800e2854379ba9325c36aed0c5e/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:56.177894 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e570d14b-f61a-4499-a955-eba712f3913d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e904abd3ae04dc3ac5aa84c69fc944e, user_id=c96f0800e2854379ba9325c36aed0c5e, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:56.181433 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e570d14b-f61a-4499-a955-eba712f3913d admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:56.195441 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 880/1749] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:56 2022] PUT /identity/v3/projects/8e904abd3ae04dc3ac5aa84c69fc944e/users/c96f0800e2854379ba9325c36aed0c5e/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:51:56.205641 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2abd75-eb15-4320-b836-ba8427cd9060 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:56.206556 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2abd75-eb15-4320-b836-ba8427cd9060 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:56.206854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2abd75-eb15-4320-b836-ba8427cd9060 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:56.237146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-ee2abd75-eb15-4320-b836-ba8427cd9060 None None] MFA Rules not processed for user `c96f0800e2854379ba9325c36aed0c5e`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:56.238373 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-ee2abd75-eb15-4320-b836-ba8427cd9060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:56.316805 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 881/1750] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:51:56.327436 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2abd75-eb15-4320-b836-ba8427cd9060 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:56.328827 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-ee2abd75-eb15-4320-b836-ba8427cd9060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:56.339323 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:56.361139 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 870/1751] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:51:56 2022] POST /identity/v3/projects => generated 375 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:51:56.368762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75681c0a-5ad3-4b6a-bd1c-932b7f56b618 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:56.428217 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2abd75-eb15-4320-b836-ba8427cd9060 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:56.448490 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75681c0a-5ad3-4b6a-bd1c-932b7f56b618 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:56.453918 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580be5f-4cc1-4cf9-b7f1-c32bde0ecd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:56.454535 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580be5f-4cc1-4cf9-b7f1-c32bde0ecd00 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:56.454816 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580be5f-4cc1-4cf9-b7f1-c32bde0ecd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:56.455112 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580be5f-4cc1-4cf9-b7f1-c32bde0ecd00 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:56.457014 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a580be5f-4cc1-4cf9-b7f1-c32bde0ecd00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:56.458674 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a580be5f-4cc1-4cf9-b7f1-c32bde0ecd00 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:56.484359 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23def9a5-97c5-4e5a-aae0-1ea72b6478b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:56.485128 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23def9a5-97c5-4e5a-aae0-1ea72b6478b9 service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:56.485470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23def9a5-97c5-4e5a-aae0-1ea72b6478b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:56.485802 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23def9a5-97c5-4e5a-aae0-1ea72b6478b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:56.488738 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-23def9a5-97c5-4e5a-aae0-1ea72b6478b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:56.554119 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23def9a5-97c5-4e5a-aae0-1ea72b6478b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:56.555582 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23def9a5-97c5-4e5a-aae0-1ea72b6478b9 service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:56.564854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 882/1752] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:51:56.583619 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 871/1753] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:51:56 2022] POST /identity/v3/users => generated 288 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:51:56.598802 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23def9a5-97c5-4e5a-aae0-1ea72b6478b9 service cinder] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:56.600984 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23def9a5-97c5-4e5a-aae0-1ea72b6478b9 service cinder] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:56.606393 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f12215-0f28-46b9-9504-1ca787daa440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:56.607015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f12215-0f28-46b9-9504-1ca787daa440 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:56.607292 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f12215-0f28-46b9-9504-1ca787daa440 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:56.607701 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f12215-0f28-46b9-9504-1ca787daa440 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:56.608658 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f12215-0f28-46b9-9504-1ca787daa440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:56.610283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f12215-0f28-46b9-9504-1ca787daa440 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:56.624076 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 883/1754] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:51:56.629581 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580be5f-4cc1-4cf9-b7f1-c32bde0ecd00 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:56.636517 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f12215-0f28-46b9-9504-1ca787daa440 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:56.655627 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f12215-0f28-46b9-9504-1ca787daa440 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:56.658433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580be5f-4cc1-4cf9-b7f1-c32bde0ecd00 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:56.660850 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf6e71c-0420-4b48-93bd-8fc52f4e237e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:56.661983 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6e71c-0420-4b48-93bd-8fc52f4e237e service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:56.662536 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e14ef8-edfb-4a6b-82e2-0f0bf5fc05cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:56.662788 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6e71c-0420-4b48-93bd-8fc52f4e237e service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:56.663176 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e14ef8-edfb-4a6b-82e2-0f0bf5fc05cc admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:56.663537 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e14ef8-edfb-4a6b-82e2-0f0bf5fc05cc admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:56.663917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e14ef8-edfb-4a6b-82e2-0f0bf5fc05cc admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:56.664407 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6e71c-0420-4b48-93bd-8fc52f4e237e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:56.664596 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e14ef8-edfb-4a6b-82e2-0f0bf5fc05cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:56.666166 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e14ef8-edfb-4a6b-82e2-0f0bf5fc05cc admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:56.666463 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-aaf6e71c-0420-4b48-93bd-8fc52f4e237e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:56.680006 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 872/1755] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:51:56.684702 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e14ef8-edfb-4a6b-82e2-0f0bf5fc05cc admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:56.699063 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e14ef8-edfb-4a6b-82e2-0f0bf5fc05cc admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:56.704952 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d86f0-5e4f-44b7-962d-dcea1e575193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:56.714004 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d86f0-5e4f-44b7-962d-dcea1e575193 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:56.714004 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d86f0-5e4f-44b7-962d-dcea1e575193 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:56.714004 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d86f0-5e4f-44b7-962d-dcea1e575193 admin admin] PATH_INFO: `/v3/projects/7bc690e6c888404fbfc488c898ad2f17/users/8cc6b901c1e847458491fd85815efc8b/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:56.723472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9d86f0-5e4f-44b7-962d-dcea1e575193 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bc690e6c888404fbfc488c898ad2f17, user_id=8cc6b901c1e847458491fd85815efc8b, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:56.726388 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9d86f0-5e4f-44b7-962d-dcea1e575193 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:56.753588 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf6e71c-0420-4b48-93bd-8fc52f4e237e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:56.755149 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf6e71c-0420-4b48-93bd-8fc52f4e237e service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:56.765087 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 884/1756] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:56 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:51:56.900160 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:56.905773 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 873/1757] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:51:56 2022] PUT /identity/v3/projects/7bc690e6c888404fbfc488c898ad2f17/users/8cc6b901c1e847458491fd85815efc8b/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:51:56.916919 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f813c5f5-fb9d-4068-a34c-84268fea2580 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:56.917297 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f813c5f5-fb9d-4068-a34c-84268fea2580 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:56.917585 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f813c5f5-fb9d-4068-a34c-84268fea2580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:56.926289 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:51:56.947956 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-f813c5f5-fb9d-4068-a34c-84268fea2580 None None] MFA Rules not processed for user `8cc6b901c1e847458491fd85815efc8b`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:51:56.949786 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-f813c5f5-fb9d-4068-a34c-84268fea2580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:51:57.038172 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 885/1758] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:51:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:51:57.376810 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d86f0-5e4f-44b7-962d-dcea1e575193 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:51:57.442413 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d86f0-5e4f-44b7-962d-dcea1e575193 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:51:57.446395 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac88aeb-cb3a-4dc1-bfd5-d9758c2d3613 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:51:57.448720 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac88aeb-cb3a-4dc1-bfd5-d9758c2d3613 service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:51:57.448720 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac88aeb-cb3a-4dc1-bfd5-d9758c2d3613 service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:51:57.448720 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac88aeb-cb3a-4dc1-bfd5-d9758c2d3613 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:51:57.457927 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac88aeb-cb3a-4dc1-bfd5-d9758c2d3613 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:51:57.459305 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac88aeb-cb3a-4dc1-bfd5-d9758c2d3613 service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:51:57.469224 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 874/1759] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:51:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:52:00.207188 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f813c5f5-fb9d-4068-a34c-84268fea2580 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:00.271172 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f813c5f5-fb9d-4068-a34c-84268fea2580 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:00.276023 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7380f-f714-451b-8722-e5a520af46e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:00.276483 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7380f-f714-451b-8722-e5a520af46e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:00.276765 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7380f-f714-451b-8722-e5a520af46e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:00.277036 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a7380f-f714-451b-8722-e5a520af46e3 admin admin] PATH_INFO: `/v3/users/4269a71ef3264e6eae68afc3a931af85` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:00.278202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a7380f-f714-451b-8722-e5a520af46e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4269a71ef3264e6eae68afc3a931af85)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:00.280171 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a7380f-f714-451b-8722-e5a520af46e3 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:00.484950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d6a7380f-f714-451b-8722-e5a520af46e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4269a71ef3264e6eae68afc3a931af85'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:00.485749 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d6a7380f-f714-451b-8722-e5a520af46e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4269a71ef3264e6eae68afc3a931af85'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:00.561369 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d6a7380f-f714-451b-8722-e5a520af46e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4269a71ef3264e6eae68afc3a931af85'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:00.582294 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d6a7380f-f714-451b-8722-e5a520af46e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4269a71ef3264e6eae68afc3a931af85'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:00.582294 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d6a7380f-f714-451b-8722-e5a520af46e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4269a71ef3264e6eae68afc3a931af85'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:00.597863 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 886/1760] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:52:00 2022] DELETE /identity/v3/users/4269a71ef3264e6eae68afc3a931af85 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:00.602268 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe157e-3841-4ea5-bd46-4d3c2287e488 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:00.602767 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe157e-3841-4ea5-bd46-4d3c2287e488 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:00.603176 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe157e-3841-4ea5-bd46-4d3c2287e488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:00.624184 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-2cfe157e-3841-4ea5-bd46-4d3c2287e488 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:00.625384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-2cfe157e-3841-4ea5-bd46-4d3c2287e488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:00.688130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 875/1761] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:00.698451 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7380f-f714-451b-8722-e5a520af46e3 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:00.699940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-d6a7380f-f714-451b-8722-e5a520af46e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:00.718722 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:00.760454 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a7380f-f714-451b-8722-e5a520af46e3 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:00.764069 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46547b92-4f6d-45a2-b3a7-7a555fa98cb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:00.764674 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46547b92-4f6d-45a2-b3a7-7a555fa98cb4 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:00.764954 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46547b92-4f6d-45a2-b3a7-7a555fa98cb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:00.765226 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46547b92-4f6d-45a2-b3a7-7a555fa98cb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:00.774672 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46547b92-4f6d-45a2-b3a7-7a555fa98cb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:00.776164 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46547b92-4f6d-45a2-b3a7-7a555fa98cb4 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:00.787032 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 887/1762] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:52:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:52:01.180333 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe157e-3841-4ea5-bd46-4d3c2287e488 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:01.181788 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-2cfe157e-3841-4ea5-bd46-4d3c2287e488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:01.199080 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe157e-3841-4ea5-bd46-4d3c2287e488 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:01.205563 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332c946-10f3-459c-911e-d20ee36cacfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:01.206178 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332c946-10f3-459c-911e-d20ee36cacfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:01.206462 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332c946-10f3-459c-911e-d20ee36cacfe admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:01.206737 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332c946-10f3-459c-911e-d20ee36cacfe admin admin] PATH_INFO: `/v3/projects/2a1b57f080dc4f648fd96a4c1f66fe24` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:01.208063 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332c946-10f3-459c-911e-d20ee36cacfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a1b57f080dc4f648fd96a4c1f66fe24)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:01.210181 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332c946-10f3-459c-911e-d20ee36cacfe admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:01.304175 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-9332c946-10f3-459c-911e-d20ee36cacfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a1b57f080dc4f648fd96a4c1f66fe24', 'request_id': 'req-9332c946-10f3-459c-911e-d20ee36cacfe'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:01.366570 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-9332c946-10f3-459c-911e-d20ee36cacfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a1b57f080dc4f648fd96a4c1f66fe24', 'request_id': 'req-9332c946-10f3-459c-911e-d20ee36cacfe'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:01.372732 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-9332c946-10f3-459c-911e-d20ee36cacfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a1b57f080dc4f648fd96a4c1f66fe24', 'request_id': 'req-9332c946-10f3-459c-911e-d20ee36cacfe'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:01.384700 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-9332c946-10f3-459c-911e-d20ee36cacfe admin admin] The token cache is being invalidate because project 2a1b57f080dc4f648fd96a4c1f66fe24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:52:01.384984 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-9332c946-10f3-459c-911e-d20ee36cacfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:01.385609 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-9332c946-10f3-459c-911e-d20ee36cacfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:01.387073 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 876/1763] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:52:01 2022] DELETE /identity/v3/projects/2a1b57f080dc4f648fd96a4c1f66fe24 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:01.391059 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46547b92-4f6d-45a2-b3a7-7a555fa98cb4 service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:01.403153 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46547b92-4f6d-45a2-b3a7-7a555fa98cb4 service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:01.458979 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecbe8cf-2386-4888-9d8e-45f42fe1ef72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:01.459601 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbe8cf-2386-4888-9d8e-45f42fe1ef72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:01.459949 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbe8cf-2386-4888-9d8e-45f42fe1ef72 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:01.460221 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecbe8cf-2386-4888-9d8e-45f42fe1ef72 admin admin] PATH_INFO: `/v3/users/c96f0800e2854379ba9325c36aed0c5e` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:01.461330 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecbe8cf-2386-4888-9d8e-45f42fe1ef72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c96f0800e2854379ba9325c36aed0c5e)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:01.463227 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecbe8cf-2386-4888-9d8e-45f42fe1ef72 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:01.678620 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-4ecbe8cf-2386-4888-9d8e-45f42fe1ef72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c96f0800e2854379ba9325c36aed0c5e'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:01.679364 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-4ecbe8cf-2386-4888-9d8e-45f42fe1ef72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c96f0800e2854379ba9325c36aed0c5e'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:01.742604 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-4ecbe8cf-2386-4888-9d8e-45f42fe1ef72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c96f0800e2854379ba9325c36aed0c5e'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:01.757149 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-4ecbe8cf-2386-4888-9d8e-45f42fe1ef72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c96f0800e2854379ba9325c36aed0c5e'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:01.763178 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-4ecbe8cf-2386-4888-9d8e-45f42fe1ef72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c96f0800e2854379ba9325c36aed0c5e'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:01.770703 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 888/1764] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:52:01 2022] DELETE /identity/v3/users/c96f0800e2854379ba9325c36aed0c5e => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:02.242384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332c946-10f3-459c-911e-d20ee36cacfe admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:02.243909 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-9332c946-10f3-459c-911e-d20ee36cacfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:02.257906 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332c946-10f3-459c-911e-d20ee36cacfe admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:02.280159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:02.317238 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0166961-0248-41a2-bdb5-84550e2ba7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:02.317874 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0166961-0248-41a2-bdb5-84550e2ba7a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:02.318152 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0166961-0248-41a2-bdb5-84550e2ba7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:02.318421 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0166961-0248-41a2-bdb5-84550e2ba7a5 admin admin] PATH_INFO: `/v3/projects/8e904abd3ae04dc3ac5aa84c69fc944e` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:02.319523 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0166961-0248-41a2-bdb5-84550e2ba7a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e904abd3ae04dc3ac5aa84c69fc944e)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:02.321494 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0166961-0248-41a2-bdb5-84550e2ba7a5 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:02.422902 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-b0166961-0248-41a2-bdb5-84550e2ba7a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e904abd3ae04dc3ac5aa84c69fc944e', 'request_id': 'req-b0166961-0248-41a2-bdb5-84550e2ba7a5'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:02.525798 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-b0166961-0248-41a2-bdb5-84550e2ba7a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e904abd3ae04dc3ac5aa84c69fc944e', 'request_id': 'req-b0166961-0248-41a2-bdb5-84550e2ba7a5'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:02.532024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-b0166961-0248-41a2-bdb5-84550e2ba7a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e904abd3ae04dc3ac5aa84c69fc944e', 'request_id': 'req-b0166961-0248-41a2-bdb5-84550e2ba7a5'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:02.543084 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-b0166961-0248-41a2-bdb5-84550e2ba7a5 admin admin] The token cache is being invalidate because project 8e904abd3ae04dc3ac5aa84c69fc944e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:52:02.543381 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-b0166961-0248-41a2-bdb5-84550e2ba7a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:02.544026 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-b0166961-0248-41a2-bdb5-84550e2ba7a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:02.545739 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 877/1765] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:52:02 2022] DELETE /identity/v3/projects/8e904abd3ae04dc3ac5aa84c69fc944e => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:02.551424 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8f38e-f5bd-4e10-8317-09fcc58bfdc7 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:02.551993 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8f38e-f5bd-4e10-8317-09fcc58bfdc7 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:02.551993 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8f38e-f5bd-4e10-8317-09fcc58bfdc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:02.572057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-1ab8f38e-f5bd-4e10-8317-09fcc58bfdc7 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:02.573252 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-1ab8f38e-f5bd-4e10-8317-09fcc58bfdc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:02.604693 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:02.634933 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 889/1766] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:02.642518 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b3485-650b-43d4-82c4-a44ee06b70a9 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:02.643839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b3485-650b-43d4-82c4-a44ee06b70a9 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:02.643839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b3485-650b-43d4-82c4-a44ee06b70a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:02.663973 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-016b3485-650b-43d4-82c4-a44ee06b70a9 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:02.665239 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-016b3485-650b-43d4-82c4-a44ee06b70a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:02.673291 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 878/1767] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:02.678173 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8f38e-f5bd-4e10-8317-09fcc58bfdc7 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:02.687992 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8f38e-f5bd-4e10-8317-09fcc58bfdc7 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:02.694159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f410d71-f485-48e6-9558-1ca053e76252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:02.694779 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f410d71-f485-48e6-9558-1ca053e76252 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:02.695060 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f410d71-f485-48e6-9558-1ca053e76252 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:02.695359 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f410d71-f485-48e6-9558-1ca053e76252 admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:02.696060 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f410d71-f485-48e6-9558-1ca053e76252 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:52:02.696365 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f410d71-f485-48e6-9558-1ca053e76252 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:02.697997 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f410d71-f485-48e6-9558-1ca053e76252 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:02.710761 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 890/1768] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:52:02.715011 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b3485-650b-43d4-82c4-a44ee06b70a9 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:02.724965 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b3485-650b-43d4-82c4-a44ee06b70a9 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:02.730029 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f256fbe-e2eb-4e9d-8879-8059f852e990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:02.730644 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f256fbe-e2eb-4e9d-8879-8059f852e990 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:02.730923 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f256fbe-e2eb-4e9d-8879-8059f852e990 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:02.731195 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f256fbe-e2eb-4e9d-8879-8059f852e990 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:02.731894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f256fbe-e2eb-4e9d-8879-8059f852e990 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:02.733777 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f256fbe-e2eb-4e9d-8879-8059f852e990 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:02.821098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 879/1769] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:52:02 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:52:02.825354 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f410d71-f485-48e6-9558-1ca053e76252 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:02.836117 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f410d71-f485-48e6-9558-1ca053e76252 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:02.902226 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa997bd6-3457-4ecf-b977-33af366ea4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:02.902876 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa997bd6-3457-4ecf-b977-33af366ea4d5 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:02.903267 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa997bd6-3457-4ecf-b977-33af366ea4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:02.903548 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa997bd6-3457-4ecf-b977-33af366ea4d5 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:02.904243 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa997bd6-3457-4ecf-b977-33af366ea4d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:02.906143 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa997bd6-3457-4ecf-b977-33af366ea4d5 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:03.001502 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 891/1770] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:52:02 2022] POST /identity/v3/users => generated 292 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:52:03.005554 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f256fbe-e2eb-4e9d-8879-8059f852e990 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:03.015962 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f256fbe-e2eb-4e9d-8879-8059f852e990 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:03.020910 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ec13d-dcd0-465c-a0d3-03f2fd1136fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:03.021499 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ec13d-dcd0-465c-a0d3-03f2fd1136fc admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:03.021775 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ec13d-dcd0-465c-a0d3-03f2fd1136fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:03.022045 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ec13d-dcd0-465c-a0d3-03f2fd1136fc admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:03.022777 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195ec13d-dcd0-465c-a0d3-03f2fd1136fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:03.024414 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195ec13d-dcd0-465c-a0d3-03f2fd1136fc admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:03.034934 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 880/1771] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:52:03.038691 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa997bd6-3457-4ecf-b977-33af366ea4d5 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:03.048655 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa997bd6-3457-4ecf-b977-33af366ea4d5 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:03.053822 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cfc873-acbf-475e-8f3b-f2da7857263f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:03.054415 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cfc873-acbf-475e-8f3b-f2da7857263f admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:03.054692 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cfc873-acbf-475e-8f3b-f2da7857263f admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:03.054964 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cfc873-acbf-475e-8f3b-f2da7857263f admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:03.055596 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cfc873-acbf-475e-8f3b-f2da7857263f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:03.057216 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cfc873-acbf-475e-8f3b-f2da7857263f admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:03.066854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 892/1772] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:52:03.070815 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ec13d-dcd0-465c-a0d3-03f2fd1136fc admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:03.080746 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ec13d-dcd0-465c-a0d3-03f2fd1136fc admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:03.086126 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e483b-0dd5-41fc-ae40-97fc673e7ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:03.086849 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e483b-0dd5-41fc-ae40-97fc673e7ac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:03.087130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e483b-0dd5-41fc-ae40-97fc673e7ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:03.087486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e483b-0dd5-41fc-ae40-97fc673e7ac5 admin admin] PATH_INFO: `/v3/projects/98c4ff300d0840669463225fdf9c9748/users/c9b914e23e8c48d5b7c5a323e14d4caf/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:03.104772 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487e483b-0dd5-41fc-ae40-97fc673e7ac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98c4ff300d0840669463225fdf9c9748, user_id=c9b914e23e8c48d5b7c5a323e14d4caf, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:03.107461 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487e483b-0dd5-41fc-ae40-97fc673e7ac5 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:03.173129 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 881/1773] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:52:03 2022] PUT /identity/v3/projects/98c4ff300d0840669463225fdf9c9748/users/c9b914e23e8c48d5b7c5a323e14d4caf/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:03.186876 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c378f4-3e42-47dd-8542-a7e2516e63a6 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:03.187088 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c378f4-3e42-47dd-8542-a7e2516e63a6 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:03.187361 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c378f4-3e42-47dd-8542-a7e2516e63a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:03.215629 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-c6c378f4-3e42-47dd-8542-a7e2516e63a6 None None] MFA Rules not processed for user `c9b914e23e8c48d5b7c5a323e14d4caf`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:03.219827 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-c6c378f4-3e42-47dd-8542-a7e2516e63a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:03.289413 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 893/1774] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:03.293884 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e483b-0dd5-41fc-ae40-97fc673e7ac5 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:03.346942 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:03.358090 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e483b-0dd5-41fc-ae40-97fc673e7ac5 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:03.362921 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af167627-eb5a-43f4-b382-04a47e27f321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:03.363522 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af167627-eb5a-43f4-b382-04a47e27f321 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:03.363829 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af167627-eb5a-43f4-b382-04a47e27f321 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:03.364114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af167627-eb5a-43f4-b382-04a47e27f321 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:03.364738 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af167627-eb5a-43f4-b382-04a47e27f321 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:03.366611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af167627-eb5a-43f4-b382-04a47e27f321 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:03.470350 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 882/1775] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:52:03 2022] POST /identity/v3/projects => generated 383 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:52:03.475105 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c378f4-3e42-47dd-8542-a7e2516e63a6 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:03.486331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c378f4-3e42-47dd-8542-a7e2516e63a6 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:03.515140 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:03.547233 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36389c47-480b-4e8c-991a-6059eb7d5fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:03.547893 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36389c47-480b-4e8c-991a-6059eb7d5fd9 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:03.548179 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36389c47-480b-4e8c-991a-6059eb7d5fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:03.548453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36389c47-480b-4e8c-991a-6059eb7d5fd9 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:03.549095 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36389c47-480b-4e8c-991a-6059eb7d5fd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:03.551016 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36389c47-480b-4e8c-991a-6059eb7d5fd9 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:03.664355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 894/1776] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:52:03 2022] POST /identity/v3/users => generated 292 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:52:03.668447 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af167627-eb5a-43f4-b382-04a47e27f321 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:03.678720 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af167627-eb5a-43f4-b382-04a47e27f321 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:03.688204 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe03f7c-6239-43f5-a642-9d2087fa3d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:03.690531 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe03f7c-6239-43f5-a642-9d2087fa3d4a admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:03.690531 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe03f7c-6239-43f5-a642-9d2087fa3d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:03.690531 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe03f7c-6239-43f5-a642-9d2087fa3d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:03.690783 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe03f7c-6239-43f5-a642-9d2087fa3d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:03.692528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe03f7c-6239-43f5-a642-9d2087fa3d4a admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:03.703670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 883/1777] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:52:03.712375 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36389c47-480b-4e8c-991a-6059eb7d5fd9 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:03.722531 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36389c47-480b-4e8c-991a-6059eb7d5fd9 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:03.727391 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad8e922-1934-4617-bb12-61ee8e593540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:03.728178 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8e922-1934-4617-bb12-61ee8e593540 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:03.728456 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8e922-1934-4617-bb12-61ee8e593540 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:03.728729 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8e922-1934-4617-bb12-61ee8e593540 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:03.729370 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad8e922-1934-4617-bb12-61ee8e593540 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:03.731102 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad8e922-1934-4617-bb12-61ee8e593540 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:03.742829 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 895/1778] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:52:03.748669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe03f7c-6239-43f5-a642-9d2087fa3d4a admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:03.757673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe03f7c-6239-43f5-a642-9d2087fa3d4a admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:03.762268 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbc77f-583b-4d82-ba9b-6dafc74f6511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:03.762922 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbc77f-583b-4d82-ba9b-6dafc74f6511 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:03.763203 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbc77f-583b-4d82-ba9b-6dafc74f6511 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:03.763483 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbc77f-583b-4d82-ba9b-6dafc74f6511 admin admin] PATH_INFO: `/v3/projects/d2c32efd670142f7b42b0162c7038751/users/5b1f8f7681bd43b3b79f6b91a4c4b8c2/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:03.782357 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbc77f-583b-4d82-ba9b-6dafc74f6511 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2c32efd670142f7b42b0162c7038751, user_id=5b1f8f7681bd43b3b79f6b91a4c4b8c2, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:03.785222 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbc77f-583b-4d82-ba9b-6dafc74f6511 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:03.891594 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 884/1779] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:52:03 2022] PUT /identity/v3/projects/d2c32efd670142f7b42b0162c7038751/users/5b1f8f7681bd43b3b79f6b91a4c4b8c2/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:52:03.902395 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069ba22-8362-423b-b838-2a7aed34e51a None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:03.903540 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069ba22-8362-423b-b838-2a7aed34e51a None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:03.911841 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069ba22-8362-423b-b838-2a7aed34e51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:03.945129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-b069ba22-8362-423b-b838-2a7aed34e51a None None] MFA Rules not processed for user `5b1f8f7681bd43b3b79f6b91a4c4b8c2`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:03.947057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-b069ba22-8362-423b-b838-2a7aed34e51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:04.043652 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 896/1780] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:04.048394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbc77f-583b-4d82-ba9b-6dafc74f6511 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:04.171723 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbc77f-583b-4d82-ba9b-6dafc74f6511 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:04.181237 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267deff2-e904-4435-94c3-719157abbc3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:04.181742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267deff2-e904-4435-94c3-719157abbc3b admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:04.182033 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267deff2-e904-4435-94c3-719157abbc3b admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:04.182345 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267deff2-e904-4435-94c3-719157abbc3b admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:04.183029 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267deff2-e904-4435-94c3-719157abbc3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:04.192175 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267deff2-e904-4435-94c3-719157abbc3b admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:04.306564 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 885/1781] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:52:04 2022] POST /identity/v3/projects => generated 383 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:52:04.316131 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069ba22-8362-423b-b838-2a7aed34e51a None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:04.329885 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069ba22-8362-423b-b838-2a7aed34e51a None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:04.372657 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267deff2-e904-4435-94c3-719157abbc3b admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:04.374638 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-267deff2-e904-4435-94c3-719157abbc3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:04.469001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267deff2-e904-4435-94c3-719157abbc3b admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:04.472701 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b8007-8d9c-4a0e-b75c-1d71a94446ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:04.472701 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b8007-8d9c-4a0e-b75c-1d71a94446ee admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:04.472701 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b8007-8d9c-4a0e-b75c-1d71a94446ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:04.472701 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b8007-8d9c-4a0e-b75c-1d71a94446ee admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:04.473160 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b8007-8d9c-4a0e-b75c-1d71a94446ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:04.474273 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3111013-88a0-4da2-9b46-e6e2286d7074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:04.474405 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b8007-8d9c-4a0e-b75c-1d71a94446ee admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:04.474915 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3111013-88a0-4da2-9b46-e6e2286d7074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:04.475199 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3111013-88a0-4da2-9b46-e6e2286d7074 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:04.475474 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3111013-88a0-4da2-9b46-e6e2286d7074 admin admin] PATH_INFO: `/v3/users/4111677cd96f4984aaae95f11fbd82cb` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:04.476711 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3111013-88a0-4da2-9b46-e6e2286d7074 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4111677cd96f4984aaae95f11fbd82cb)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:04.478747 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3111013-88a0-4da2-9b46-e6e2286d7074 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:04.555440 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 897/1782] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:52:04 2022] POST /identity/v3/users => generated 298 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:52:04.559428 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b8007-8d9c-4a0e-b75c-1d71a94446ee admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:04.569570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b8007-8d9c-4a0e-b75c-1d71a94446ee admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:04.574065 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1439b8-fefb-4854-9755-12cd8fba1ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:04.574637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1439b8-fefb-4854-9755-12cd8fba1ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:04.574916 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1439b8-fefb-4854-9755-12cd8fba1ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:04.575186 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1439b8-fefb-4854-9755-12cd8fba1ff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:04.575841 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1439b8-fefb-4854-9755-12cd8fba1ff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:04.577435 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1439b8-fefb-4854-9755-12cd8fba1ff2 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:04.591188 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 898/1783] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:52:04.591725 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:04.592515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1439b8-fefb-4854-9755-12cd8fba1ff2 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:04.602502 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1439b8-fefb-4854-9755-12cd8fba1ff2 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:04.607198 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766d317-125b-4d84-9231-e8cca4264d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:04.607893 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766d317-125b-4d84-9231-e8cca4264d5a admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:04.608176 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766d317-125b-4d84-9231-e8cca4264d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:04.608452 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766d317-125b-4d84-9231-e8cca4264d5a admin admin] PATH_INFO: `/v3/projects/cda4c94a481945b9b19aaf41135a3bcd/users/433d86187dbd4cfd88e5cd3536bf5c85/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:04.623304 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7766d317-125b-4d84-9231-e8cca4264d5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cda4c94a481945b9b19aaf41135a3bcd, user_id=433d86187dbd4cfd88e5cd3536bf5c85, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:04.626173 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7766d317-125b-4d84-9231-e8cca4264d5a admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:04.628900 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:04.698710 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c3111013-88a0-4da2-9b46-e6e2286d7074 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4111677cd96f4984aaae95f11fbd82cb'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:04.715473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 899/1784] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:52:04 2022] PUT /identity/v3/projects/cda4c94a481945b9b19aaf41135a3bcd/users/433d86187dbd4cfd88e5cd3536bf5c85/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:52:04.719185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766d317-125b-4d84-9231-e8cca4264d5a admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:04.767030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c3111013-88a0-4da2-9b46-e6e2286d7074 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4111677cd96f4984aaae95f11fbd82cb'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:04.782175 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c3111013-88a0-4da2-9b46-e6e2286d7074 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4111677cd96f4984aaae95f11fbd82cb'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:04.782403 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766d317-125b-4d84-9231-e8cca4264d5a admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:04.782955 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c3111013-88a0-4da2-9b46-e6e2286d7074 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4111677cd96f4984aaae95f11fbd82cb'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:04.783545 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c3111013-88a0-4da2-9b46-e6e2286d7074 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4111677cd96f4984aaae95f11fbd82cb'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:04.786982 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe8958-686a-4ef0-8c81-3e6003646ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:04.787594 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe8958-686a-4ef0-8c81-3e6003646ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:04.787913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe8958-686a-4ef0-8c81-3e6003646ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:04.788187 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe8958-686a-4ef0-8c81-3e6003646ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:04.788822 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe8958-686a-4ef0-8c81-3e6003646ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:04.790399 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe8958-686a-4ef0-8c81-3e6003646ea3 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:04.796300 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 886/1785] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:52:04 2022] DELETE /identity/v3/users/4111677cd96f4984aaae95f11fbd82cb => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:04.801433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51800c-9bcd-41af-a3b6-dc3d14e46517 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:04.801714 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51800c-9bcd-41af-a3b6-dc3d14e46517 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:04.801983 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51800c-9bcd-41af-a3b6-dc3d14e46517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:04.804746 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 900/1786] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:52:04 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:52:04.813559 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe8958-686a-4ef0-8c81-3e6003646ea3 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:04.815913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-c6fe8958-686a-4ef0-8c81-3e6003646ea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:04.822640 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-1d51800c-9bcd-41af-a3b6-dc3d14e46517 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:04.823845 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-1d51800c-9bcd-41af-a3b6-dc3d14e46517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:04.828593 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe8958-686a-4ef0-8c81-3e6003646ea3 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:04.888928 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 887/1787] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:52:04.889386 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f39ee0-b30d-466f-b96e-588102f30db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:04.890082 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f39ee0-b30d-466f-b96e-588102f30db9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:04.890487 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f39ee0-b30d-466f-b96e-588102f30db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:04.890708 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f39ee0-b30d-466f-b96e-588102f30db9 admin admin] PATH_INFO: `/v3/projects/cda4c94a481945b9b19aaf41135a3bcd/users/433d86187dbd4cfd88e5cd3536bf5c85/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:04.892677 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f39ee0-b30d-466f-b96e-588102f30db9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cda4c94a481945b9b19aaf41135a3bcd, user_id=433d86187dbd4cfd88e5cd3536bf5c85, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:04.896565 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51800c-9bcd-41af-a3b6-dc3d14e46517 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:04.896677 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f39ee0-b30d-466f-b96e-588102f30db9 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:04.896765 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-1d51800c-9bcd-41af-a3b6-dc3d14e46517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:04.906497 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 901/1788] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:52:04 2022] PUT /identity/v3/projects/cda4c94a481945b9b19aaf41135a3bcd/users/433d86187dbd4cfd88e5cd3536bf5c85/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:52:04.917829 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c95e8-4ce4-47cc-a4ad-e3139881c0f6 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:04.918197 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c95e8-4ce4-47cc-a4ad-e3139881c0f6 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:04.918644 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c95e8-4ce4-47cc-a4ad-e3139881c0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:04.948544 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-369c95e8-4ce4-47cc-a4ad-e3139881c0f6 None None] MFA Rules not processed for user `433d86187dbd4cfd88e5cd3536bf5c85`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:04.949746 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-369c95e8-4ce4-47cc-a4ad-e3139881c0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:04.953893 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51800c-9bcd-41af-a3b6-dc3d14e46517 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:05.006331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b1bff5-c17b-49bb-bae2-c088c33bee4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:05.007051 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1bff5-c17b-49bb-bae2-c088c33bee4b service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:05.007328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1bff5-c17b-49bb-bae2-c088c33bee4b service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:05.007597 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1bff5-c17b-49bb-bae2-c088c33bee4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:05.021711 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 902/1789] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:04 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:52:05.033426 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369c95e8-4ce4-47cc-a4ad-e3139881c0f6 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:05.046155 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-369c95e8-4ce4-47cc-a4ad-e3139881c0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:05.051779 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:05.075047 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b1bff5-c17b-49bb-bae2-c088c33bee4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:05.076587 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b1bff5-c17b-49bb-bae2-c088c33bee4b service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:05.086949 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 888/1790] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:52:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:52:05.091446 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:05.101461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369c95e8-4ce4-47cc-a4ad-e3139881c0f6 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:05.105395 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f24f21-ef8a-4f85-b473-9322df7c14c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:05.106036 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f24f21-ef8a-4f85-b473-9322df7c14c7 service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:05.106322 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f24f21-ef8a-4f85-b473-9322df7c14c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:05.106622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f24f21-ef8a-4f85-b473-9322df7c14c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:05.108531 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-47f24f21-ef8a-4f85-b473-9322df7c14c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:05.177879 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f24f21-ef8a-4f85-b473-9322df7c14c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:05.179322 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f24f21-ef8a-4f85-b473-9322df7c14c7 service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:05.189353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 903/1791] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:52:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:52:05.528549 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b1bff5-c17b-49bb-bae2-c088c33bee4b service neutron] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:05.529988 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-07b1bff5-c17b-49bb-bae2-c088c33bee4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:05.550682 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b1bff5-c17b-49bb-bae2-c088c33bee4b service neutron] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:05.562584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28683b97-e400-4271-b0f7-4b88682aeeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:05.563255 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28683b97-e400-4271-b0f7-4b88682aeeb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:05.563583 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28683b97-e400-4271-b0f7-4b88682aeeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:05.563924 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28683b97-e400-4271-b0f7-4b88682aeeb2 admin admin] PATH_INFO: `/v3/projects/5828f1a4f3e04eb0acde9539b1acf0e5` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:05.567421 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28683b97-e400-4271-b0f7-4b88682aeeb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5828f1a4f3e04eb0acde9539b1acf0e5)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:05.569372 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28683b97-e400-4271-b0f7-4b88682aeeb2 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:05.660078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-28683b97-e400-4271-b0f7-4b88682aeeb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5828f1a4f3e04eb0acde9539b1acf0e5', 'request_id': 'req-28683b97-e400-4271-b0f7-4b88682aeeb2'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:05.762926 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-28683b97-e400-4271-b0f7-4b88682aeeb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5828f1a4f3e04eb0acde9539b1acf0e5', 'request_id': 'req-28683b97-e400-4271-b0f7-4b88682aeeb2'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:05.773196 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-28683b97-e400-4271-b0f7-4b88682aeeb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5828f1a4f3e04eb0acde9539b1acf0e5', 'request_id': 'req-28683b97-e400-4271-b0f7-4b88682aeeb2'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:05.790111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-28683b97-e400-4271-b0f7-4b88682aeeb2 admin admin] The token cache is being invalidate because project 5828f1a4f3e04eb0acde9539b1acf0e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:52:05.790399 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-28683b97-e400-4271-b0f7-4b88682aeeb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:05.791106 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-28683b97-e400-4271-b0f7-4b88682aeeb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:05.792645 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 889/1792] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:52:05 2022] DELETE /identity/v3/projects/5828f1a4f3e04eb0acde9539b1acf0e5 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:05.809368 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f24f21-ef8a-4f85-b473-9322df7c14c7 service cinder] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:05.815754 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f24f21-ef8a-4f85-b473-9322df7c14c7 service cinder] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:05.875460 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b6ca2-1fee-4887-8622-863f76cbaa96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:05.876196 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b6ca2-1fee-4887-8622-863f76cbaa96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:05.876472 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b6ca2-1fee-4887-8622-863f76cbaa96 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:05.876747 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b6ca2-1fee-4887-8622-863f76cbaa96 admin admin] PATH_INFO: `/v3/users/17fe64aa8ed04d1bb29f8ce42d898e3e` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:05.878351 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767b6ca2-1fee-4887-8622-863f76cbaa96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17fe64aa8ed04d1bb29f8ce42d898e3e)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:05.880318 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767b6ca2-1fee-4887-8622-863f76cbaa96 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:06.076501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-767b6ca2-1fee-4887-8622-863f76cbaa96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17fe64aa8ed04d1bb29f8ce42d898e3e'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:06.078850 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-767b6ca2-1fee-4887-8622-863f76cbaa96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17fe64aa8ed04d1bb29f8ce42d898e3e'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:06.155424 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-767b6ca2-1fee-4887-8622-863f76cbaa96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17fe64aa8ed04d1bb29f8ce42d898e3e'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:06.164157 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:06.168595 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-767b6ca2-1fee-4887-8622-863f76cbaa96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17fe64aa8ed04d1bb29f8ce42d898e3e'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:06.169263 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-767b6ca2-1fee-4887-8622-863f76cbaa96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17fe64aa8ed04d1bb29f8ce42d898e3e'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:06.181233 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 904/1793] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:52:05 2022] DELETE /identity/v3/users/17fe64aa8ed04d1bb29f8ce42d898e3e => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:06.601921 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28683b97-e400-4271-b0f7-4b88682aeeb2 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:06.603333 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-28683b97-e400-4271-b0f7-4b88682aeeb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:06.616852 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28683b97-e400-4271-b0f7-4b88682aeeb2 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:06.675397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3251482-aa92-46e3-9a1b-70507df54c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:06.676061 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3251482-aa92-46e3-9a1b-70507df54c99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:06.676340 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3251482-aa92-46e3-9a1b-70507df54c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:06.676609 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3251482-aa92-46e3-9a1b-70507df54c99 admin admin] PATH_INFO: `/v3/projects/ed29e0adff8a48feab6eb3b7447d9532` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:06.677752 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3251482-aa92-46e3-9a1b-70507df54c99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed29e0adff8a48feab6eb3b7447d9532)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:06.679629 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3251482-aa92-46e3-9a1b-70507df54c99 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:06.774209 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:06.782584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c3251482-aa92-46e3-9a1b-70507df54c99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed29e0adff8a48feab6eb3b7447d9532', 'request_id': 'req-c3251482-aa92-46e3-9a1b-70507df54c99'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:06.852125 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c3251482-aa92-46e3-9a1b-70507df54c99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed29e0adff8a48feab6eb3b7447d9532', 'request_id': 'req-c3251482-aa92-46e3-9a1b-70507df54c99'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:06.858501 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c3251482-aa92-46e3-9a1b-70507df54c99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed29e0adff8a48feab6eb3b7447d9532', 'request_id': 'req-c3251482-aa92-46e3-9a1b-70507df54c99'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:06.870622 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c3251482-aa92-46e3-9a1b-70507df54c99 admin admin] The token cache is being invalidate because project ed29e0adff8a48feab6eb3b7447d9532 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:52:06.870890 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c3251482-aa92-46e3-9a1b-70507df54c99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:06.871523 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c3251482-aa92-46e3-9a1b-70507df54c99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:06.873087 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 890/1794] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:52:06 2022] DELETE /identity/v3/projects/ed29e0adff8a48feab6eb3b7447d9532 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:06.878071 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24168e0-f7e7-42b4-b688-e5ad33e05ca4 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:06.878518 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24168e0-f7e7-42b4-b688-e5ad33e05ca4 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:06.878701 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24168e0-f7e7-42b4-b688-e5ad33e05ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:06.897861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-b24168e0-f7e7-42b4-b688-e5ad33e05ca4 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:06.899041 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-b24168e0-f7e7-42b4-b688-e5ad33e05ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:06.957513 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 905/1795] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:06.965130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75493064-88eb-4f24-890b-c07f2e37f50f None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:06.965422 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75493064-88eb-4f24-890b-c07f2e37f50f None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:06.965691 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75493064-88eb-4f24-890b-c07f2e37f50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:06.985249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-75493064-88eb-4f24-890b-c07f2e37f50f None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:06.987040 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-75493064-88eb-4f24-890b-c07f2e37f50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:06.994809 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 891/1796] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:06.998889 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24168e0-f7e7-42b4-b688-e5ad33e05ca4 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:07.008586 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24168e0-f7e7-42b4-b688-e5ad33e05ca4 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:07.013117 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb4435-d2b4-427c-bc26-91e102c00374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:07.013718 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb4435-d2b4-427c-bc26-91e102c00374 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:07.013993 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb4435-d2b4-427c-bc26-91e102c00374 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:07.014263 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb4435-d2b4-427c-bc26-91e102c00374 admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:07.014921 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb4435-d2b4-427c-bc26-91e102c00374 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:52:07.015276 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb4435-d2b4-427c-bc26-91e102c00374 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:07.016937 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb4435-d2b4-427c-bc26-91e102c00374 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:07.028861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 906/1797] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:52:07.032871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75493064-88eb-4f24-890b-c07f2e37f50f None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:07.042614 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75493064-88eb-4f24-890b-c07f2e37f50f None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:07.047162 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1066cd-1aa2-477d-8767-bbcd83224473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:07.047773 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1066cd-1aa2-477d-8767-bbcd83224473 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:07.048062 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1066cd-1aa2-477d-8767-bbcd83224473 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:07.048332 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1066cd-1aa2-477d-8767-bbcd83224473 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:07.048969 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1066cd-1aa2-477d-8767-bbcd83224473 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:07.050840 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1066cd-1aa2-477d-8767-bbcd83224473 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:07.183233 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 892/1798] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:52:07 2022] POST /identity/v3/projects => generated 383 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:52:07.187380 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb4435-d2b4-427c-bc26-91e102c00374 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:07.197359 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb4435-d2b4-427c-bc26-91e102c00374 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:07.253457 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db8a34-fe40-4947-aa9e-3a14c9b65ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:07.254062 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db8a34-fe40-4947-aa9e-3a14c9b65ae9 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:07.254348 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db8a34-fe40-4947-aa9e-3a14c9b65ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:07.254624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76db8a34-fe40-4947-aa9e-3a14c9b65ae9 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:07.255263 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db8a34-fe40-4947-aa9e-3a14c9b65ae9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:07.257213 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76db8a34-fe40-4947-aa9e-3a14c9b65ae9 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:07.340881 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 907/1799] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:52:07 2022] POST /identity/v3/users => generated 292 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:52:07.342106 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1066cd-1aa2-477d-8767-bbcd83224473 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:07.352734 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1066cd-1aa2-477d-8767-bbcd83224473 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:07.357318 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600dfc82-b946-47bf-bacf-f9b2f6ede787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:07.357906 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600dfc82-b946-47bf-bacf-f9b2f6ede787 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:07.358183 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600dfc82-b946-47bf-bacf-f9b2f6ede787 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:07.358454 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600dfc82-b946-47bf-bacf-f9b2f6ede787 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:07.359087 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600dfc82-b946-47bf-bacf-f9b2f6ede787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:07.360691 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600dfc82-b946-47bf-bacf-f9b2f6ede787 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:07.370932 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 893/1800] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:52:07.374778 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db8a34-fe40-4947-aa9e-3a14c9b65ae9 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:07.384617 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76db8a34-fe40-4947-aa9e-3a14c9b65ae9 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:07.389227 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce135f3-b5cf-4231-bf8f-4809040d5b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:07.389818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce135f3-b5cf-4231-bf8f-4809040d5b92 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:07.390100 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce135f3-b5cf-4231-bf8f-4809040d5b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:07.390384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce135f3-b5cf-4231-bf8f-4809040d5b92 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:07.391012 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce135f3-b5cf-4231-bf8f-4809040d5b92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:07.392616 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce135f3-b5cf-4231-bf8f-4809040d5b92 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:07.402961 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 908/1801] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:52:07.406947 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600dfc82-b946-47bf-bacf-f9b2f6ede787 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:07.417368 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600dfc82-b946-47bf-bacf-f9b2f6ede787 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:07.421942 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad4901-7db3-41d9-af4c-58c66612c27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:07.422586 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad4901-7db3-41d9-af4c-58c66612c27d admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:07.422872 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad4901-7db3-41d9-af4c-58c66612c27d admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:07.423145 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad4901-7db3-41d9-af4c-58c66612c27d admin admin] PATH_INFO: `/v3/projects/058f0160f3b544bb96aa73081b9ee948/users/12e2e32b7320405486dd065ad8db20c7/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:07.437991 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad4901-7db3-41d9-af4c-58c66612c27d admin admin] RBAC: Authorizing `identity:create_grant(project_id=058f0160f3b544bb96aa73081b9ee948, user_id=12e2e32b7320405486dd065ad8db20c7, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:07.440616 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad4901-7db3-41d9-af4c-58c66612c27d admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:07.509165 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 894/1802] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:52:07 2022] PUT /identity/v3/projects/058f0160f3b544bb96aa73081b9ee948/users/12e2e32b7320405486dd065ad8db20c7/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:07.519658 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b99232-95d6-4824-8894-68d4f7684cc2 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:07.519974 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b99232-95d6-4824-8894-68d4f7684cc2 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:07.520247 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b99232-95d6-4824-8894-68d4f7684cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:07.549462 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-b7b99232-95d6-4824-8894-68d4f7684cc2 None None] MFA Rules not processed for user `12e2e32b7320405486dd065ad8db20c7`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:07.551347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-b7b99232-95d6-4824-8894-68d4f7684cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:07.590239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:07.617425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 909/1803] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:07.622148 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad4901-7db3-41d9-af4c-58c66612c27d admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:07.694784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad4901-7db3-41d9-af4c-58c66612c27d admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:07.699447 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acd017-8a75-4403-8e8c-0684fccab755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:07.700092 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acd017-8a75-4403-8e8c-0684fccab755 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:07.700381 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acd017-8a75-4403-8e8c-0684fccab755 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:07.700659 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10acd017-8a75-4403-8e8c-0684fccab755 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:07.701305 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10acd017-8a75-4403-8e8c-0684fccab755 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:07.703191 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10acd017-8a75-4403-8e8c-0684fccab755 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:07.774347 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 895/1804] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:52:07 2022] POST /identity/v3/projects => generated 383 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:52:07.778560 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b99232-95d6-4824-8894-68d4f7684cc2 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:07.790293 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b99232-95d6-4824-8894-68d4f7684cc2 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:07.851190 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77edd0-19d1-411d-b31f-d7a7930c3b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:07.851930 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77edd0-19d1-411d-b31f-d7a7930c3b15 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:07.852211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77edd0-19d1-411d-b31f-d7a7930c3b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:07.852480 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77edd0-19d1-411d-b31f-d7a7930c3b15 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:07.853105 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d77edd0-19d1-411d-b31f-d7a7930c3b15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:07.855102 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d77edd0-19d1-411d-b31f-d7a7930c3b15 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:07.965671 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 910/1805] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:52:07 2022] POST /identity/v3/users => generated 298 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:52:07.969709 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acd017-8a75-4403-8e8c-0684fccab755 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:07.981386 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10acd017-8a75-4403-8e8c-0684fccab755 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:07.986135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74150165-12e5-479f-934f-743c3a5cf1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:07.986928 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74150165-12e5-479f-934f-743c3a5cf1e8 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:07.987299 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74150165-12e5-479f-934f-743c3a5cf1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:07.987517 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74150165-12e5-479f-934f-743c3a5cf1e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:07.988199 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74150165-12e5-479f-934f-743c3a5cf1e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:07.991657 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74150165-12e5-479f-934f-743c3a5cf1e8 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:08.000991 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 896/1806] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:52:08.005221 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77edd0-19d1-411d-b31f-d7a7930c3b15 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:08.015201 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77edd0-19d1-411d-b31f-d7a7930c3b15 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:08.020235 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe068a60-6fc8-4a7e-842a-465f78d08fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:08.020868 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe068a60-6fc8-4a7e-842a-465f78d08fa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:08.021144 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe068a60-6fc8-4a7e-842a-465f78d08fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:08.021414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe068a60-6fc8-4a7e-842a-465f78d08fa7 admin admin] PATH_INFO: `/v3/projects/d9ebd05b5afc42028af1064478e9f5fc/users/eac13cb0d0dd412cbbd50ecc46fee071/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:08.037736 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe068a60-6fc8-4a7e-842a-465f78d08fa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ebd05b5afc42028af1064478e9f5fc, user_id=eac13cb0d0dd412cbbd50ecc46fee071, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:08.040807 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe068a60-6fc8-4a7e-842a-465f78d08fa7 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:08.114348 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 911/1807] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:52:08 2022] PUT /identity/v3/projects/d9ebd05b5afc42028af1064478e9f5fc/users/eac13cb0d0dd412cbbd50ecc46fee071/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:08.118764 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74150165-12e5-479f-934f-743c3a5cf1e8 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:08.129742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74150165-12e5-479f-934f-743c3a5cf1e8 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:08.163917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:08.187603 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400fbd6-8a39-458e-84bb-472618dbedd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:08.188252 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400fbd6-8a39-458e-84bb-472618dbedd5 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:08.188565 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400fbd6-8a39-458e-84bb-472618dbedd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:08.188824 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400fbd6-8a39-458e-84bb-472618dbedd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:08.189958 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400fbd6-8a39-458e-84bb-472618dbedd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:08.191564 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400fbd6-8a39-458e-84bb-472618dbedd5 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:08.201546 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 897/1808] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:52:08.206632 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe068a60-6fc8-4a7e-842a-465f78d08fa7 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:08.218241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe068a60-6fc8-4a7e-842a-465f78d08fa7 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:08.223107 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f4bbf3-6e55-4fc6-80f8-f7343d2b4705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:08.223800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4bbf3-6e55-4fc6-80f8-f7343d2b4705 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:08.224087 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4bbf3-6e55-4fc6-80f8-f7343d2b4705 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:08.224358 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f4bbf3-6e55-4fc6-80f8-f7343d2b4705 admin admin] PATH_INFO: `/v3/projects/d9ebd05b5afc42028af1064478e9f5fc/users/eac13cb0d0dd412cbbd50ecc46fee071/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:08.226251 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f4bbf3-6e55-4fc6-80f8-f7343d2b4705 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ebd05b5afc42028af1064478e9f5fc, user_id=eac13cb0d0dd412cbbd50ecc46fee071, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:08.228918 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f4bbf3-6e55-4fc6-80f8-f7343d2b4705 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:08.239479 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 912/1809] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:52:08 2022] PUT /identity/v3/projects/d9ebd05b5afc42028af1064478e9f5fc/users/eac13cb0d0dd412cbbd50ecc46fee071/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:08.251359 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139f714-9680-47dc-8af4-52a5dc133047 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:08.251720 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139f714-9680-47dc-8af4-52a5dc133047 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:08.252526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139f714-9680-47dc-8af4-52a5dc133047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:08.281780 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-a139f714-9680-47dc-8af4-52a5dc133047 None None] MFA Rules not processed for user `eac13cb0d0dd412cbbd50ecc46fee071`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:08.283000 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-a139f714-9680-47dc-8af4-52a5dc133047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:08.353773 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 898/1810] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:08 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:08.366440 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f4bbf3-6e55-4fc6-80f8-f7343d2b4705 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:08.368495 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-49f4bbf3-6e55-4fc6-80f8-f7343d2b4705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:08.428350 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f4bbf3-6e55-4fc6-80f8-f7343d2b4705 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:08.432036 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e3fc8-b734-4021-abf6-8ef796b11a44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:08.432640 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e3fc8-b734-4021-abf6-8ef796b11a44 service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:08.432915 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e3fc8-b734-4021-abf6-8ef796b11a44 service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:08.433187 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e3fc8-b734-4021-abf6-8ef796b11a44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:08.442737 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e3fc8-b734-4021-abf6-8ef796b11a44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:08.444183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e3fc8-b734-4021-abf6-8ef796b11a44 service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:08.453755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 913/1811] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:52:08 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:52:08.566270 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a139f714-9680-47dc-8af4-52a5dc133047 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:08.577338 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a139f714-9680-47dc-8af4-52a5dc133047 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:08.581441 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec2ff1-fa8d-494f-acd6-0d6cd86d0484 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:08.582074 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec2ff1-fa8d-494f-acd6-0d6cd86d0484 service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:08.582410 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec2ff1-fa8d-494f-acd6-0d6cd86d0484 service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:08.582857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ec2ff1-fa8d-494f-acd6-0d6cd86d0484 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:08.640139 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec2ff1-fa8d-494f-acd6-0d6cd86d0484 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:08.641574 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ec2ff1-fa8d-494f-acd6-0d6cd86d0484 service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:08.650394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 899/1812] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:52:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:52:09.597198 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e3fc8-b734-4021-abf6-8ef796b11a44 service cinder] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:09.606004 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e3fc8-b734-4021-abf6-8ef796b11a44 service cinder] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:09.609852 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd03c3-f057-4350-9edc-ce0e17f4d28a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:09.612088 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd03c3-f057-4350-9edc-ce0e17f4d28a service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:09.612088 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd03c3-f057-4350-9edc-ce0e17f4d28a service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:09.612088 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd03c3-f057-4350-9edc-ce0e17f4d28a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:09.612955 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-dcdd03c3-f057-4350-9edc-ce0e17f4d28a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:09.646441 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:09.674621 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd03c3-f057-4350-9edc-ce0e17f4d28a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:09.676194 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd03c3-f057-4350-9edc-ce0e17f4d28a service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:09.688475 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 914/1813] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:52:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:52:11.153939 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec2ff1-fa8d-494f-acd6-0d6cd86d0484 service cinder] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:11.157655 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-10ec2ff1-fa8d-494f-acd6-0d6cd86d0484 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:11.167931 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ec2ff1-fa8d-494f-acd6-0d6cd86d0484 service cinder] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:11.171991 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f346d-184d-4300-9531-8694fc1daedf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:11.172607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f346d-184d-4300-9531-8694fc1daedf service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:11.173002 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f346d-184d-4300-9531-8694fc1daedf service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:11.177794 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f346d-184d-4300-9531-8694fc1daedf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:11.177794 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-1d0f346d-184d-4300-9531-8694fc1daedf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:11.234236 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:11.246717 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f346d-184d-4300-9531-8694fc1daedf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:11.248315 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f346d-184d-4300-9531-8694fc1daedf service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:11.258428 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 900/1814] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:52:11 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:52:11.951566 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd03c3-f057-4350-9edc-ce0e17f4d28a service cinder] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:12.024646 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd03c3-f057-4350-9edc-ce0e17f4d28a service cinder] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:12.029741 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870a64a-7a00-4b7b-b278-4ce9b91cba75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:12.030956 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870a64a-7a00-4b7b-b278-4ce9b91cba75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:12.031243 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870a64a-7a00-4b7b-b278-4ce9b91cba75 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:12.031522 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870a64a-7a00-4b7b-b278-4ce9b91cba75 admin admin] PATH_INFO: `/v3/users/12e2e32b7320405486dd065ad8db20c7` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:12.032721 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9870a64a-7a00-4b7b-b278-4ce9b91cba75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12e2e32b7320405486dd065ad8db20c7)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:12.035398 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9870a64a-7a00-4b7b-b278-4ce9b91cba75 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:12.265678 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-9870a64a-7a00-4b7b-b278-4ce9b91cba75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12e2e32b7320405486dd065ad8db20c7'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:12.266795 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-9870a64a-7a00-4b7b-b278-4ce9b91cba75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12e2e32b7320405486dd065ad8db20c7'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:12.338337 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-9870a64a-7a00-4b7b-b278-4ce9b91cba75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12e2e32b7320405486dd065ad8db20c7'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:12.363520 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-9870a64a-7a00-4b7b-b278-4ce9b91cba75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12e2e32b7320405486dd065ad8db20c7'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:12.365869 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-9870a64a-7a00-4b7b-b278-4ce9b91cba75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12e2e32b7320405486dd065ad8db20c7'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:12.388210 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 915/1815] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:52:11 2022] DELETE /identity/v3/users/12e2e32b7320405486dd065ad8db20c7 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:12.393022 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbccbdf-34ef-4031-90e0-b82aa4f2a571 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:12.394025 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbccbdf-34ef-4031-90e0-b82aa4f2a571 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:12.394565 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbccbdf-34ef-4031-90e0-b82aa4f2a571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:12.415886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-0fbccbdf-34ef-4031-90e0-b82aa4f2a571 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:12.417225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-0fbccbdf-34ef-4031-90e0-b82aa4f2a571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:12.492366 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 901/1816] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:12.506701 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870a64a-7a00-4b7b-b278-4ce9b91cba75 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:12.509680 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-9870a64a-7a00-4b7b-b278-4ce9b91cba75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:12.577123 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870a64a-7a00-4b7b-b278-4ce9b91cba75 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:12.581833 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041b411-27f0-49d0-a719-204ebaf79b4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:12.582460 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041b411-27f0-49d0-a719-204ebaf79b4f service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:12.582739 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041b411-27f0-49d0-a719-204ebaf79b4f service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:12.586101 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041b411-27f0-49d0-a719-204ebaf79b4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:12.589249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:12.594082 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e041b411-27f0-49d0-a719-204ebaf79b4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:12.595664 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e041b411-27f0-49d0-a719-204ebaf79b4f service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:12.606976 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 916/1817] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:52:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:52:13.279409 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbccbdf-34ef-4031-90e0-b82aa4f2a571 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:13.280876 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-0fbccbdf-34ef-4031-90e0-b82aa4f2a571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:13.302156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbccbdf-34ef-4031-90e0-b82aa4f2a571 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:13.320136 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a1485-172b-447d-b930-9c3d0615e45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:13.321071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a1485-172b-447d-b930-9c3d0615e45c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:13.321499 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a1485-172b-447d-b930-9c3d0615e45c admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:13.322040 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a1485-172b-447d-b930-9c3d0615e45c admin admin] PATH_INFO: `/v3/projects/058f0160f3b544bb96aa73081b9ee948` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:13.323359 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53a1485-172b-447d-b930-9c3d0615e45c admin admin] RBAC: Authorizing `identity:delete_project(project_id=058f0160f3b544bb96aa73081b9ee948)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:13.327108 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53a1485-172b-447d-b930-9c3d0615e45c admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:13.464226 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-d53a1485-172b-447d-b930-9c3d0615e45c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '058f0160f3b544bb96aa73081b9ee948', 'request_id': 'req-d53a1485-172b-447d-b930-9c3d0615e45c'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:13.538141 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-d53a1485-172b-447d-b930-9c3d0615e45c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '058f0160f3b544bb96aa73081b9ee948', 'request_id': 'req-d53a1485-172b-447d-b930-9c3d0615e45c'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:13.544691 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-d53a1485-172b-447d-b930-9c3d0615e45c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '058f0160f3b544bb96aa73081b9ee948', 'request_id': 'req-d53a1485-172b-447d-b930-9c3d0615e45c'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:13.564664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-d53a1485-172b-447d-b930-9c3d0615e45c admin admin] The token cache is being invalidate because project 058f0160f3b544bb96aa73081b9ee948 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:52:13.572083 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-d53a1485-172b-447d-b930-9c3d0615e45c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:13.574848 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-d53a1485-172b-447d-b930-9c3d0615e45c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:13.578307 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 902/1818] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:52:13 2022] DELETE /identity/v3/projects/058f0160f3b544bb96aa73081b9ee948 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:13.582769 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041b411-27f0-49d0-a719-204ebaf79b4f service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:13.594420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041b411-27f0-49d0-a719-204ebaf79b4f service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:13.708617 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5001bc-96a3-4ee0-8afa-dbab546f4fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:13.709152 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5001bc-96a3-4ee0-8afa-dbab546f4fc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:13.709433 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5001bc-96a3-4ee0-8afa-dbab546f4fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:13.709942 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5001bc-96a3-4ee0-8afa-dbab546f4fc1 admin admin] PATH_INFO: `/v3/users/eac13cb0d0dd412cbbd50ecc46fee071` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:13.711132 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5001bc-96a3-4ee0-8afa-dbab546f4fc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eac13cb0d0dd412cbbd50ecc46fee071)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:13.713178 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5001bc-96a3-4ee0-8afa-dbab546f4fc1 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:13.961184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-7a5001bc-96a3-4ee0-8afa-dbab546f4fc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eac13cb0d0dd412cbbd50ecc46fee071'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:13.962322 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-7a5001bc-96a3-4ee0-8afa-dbab546f4fc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eac13cb0d0dd412cbbd50ecc46fee071'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:14.045363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-7a5001bc-96a3-4ee0-8afa-dbab546f4fc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eac13cb0d0dd412cbbd50ecc46fee071'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:14.063285 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-7a5001bc-96a3-4ee0-8afa-dbab546f4fc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eac13cb0d0dd412cbbd50ecc46fee071'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:14.064234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-7a5001bc-96a3-4ee0-8afa-dbab546f4fc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eac13cb0d0dd412cbbd50ecc46fee071'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:14.086690 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 917/1819] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:52:13 2022] DELETE /identity/v3/users/eac13cb0d0dd412cbbd50ecc46fee071 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:14.558022 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a1485-172b-447d-b930-9c3d0615e45c admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:14.559730 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-d53a1485-172b-447d-b930-9c3d0615e45c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:14.585844 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a1485-172b-447d-b930-9c3d0615e45c admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:14.666788 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e194bc-4175-4b70-978b-6f553c17d15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:14.667629 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e194bc-4175-4b70-978b-6f553c17d15f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:14.667831 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e194bc-4175-4b70-978b-6f553c17d15f admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:14.668181 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e194bc-4175-4b70-978b-6f553c17d15f admin admin] PATH_INFO: `/v3/projects/d9ebd05b5afc42028af1064478e9f5fc` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:14.669491 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e194bc-4175-4b70-978b-6f553c17d15f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9ebd05b5afc42028af1064478e9f5fc)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:14.671641 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e194bc-4175-4b70-978b-6f553c17d15f admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:14.689532 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:14.819556 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-a1e194bc-4175-4b70-978b-6f553c17d15f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9ebd05b5afc42028af1064478e9f5fc', 'request_id': 'req-a1e194bc-4175-4b70-978b-6f553c17d15f'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:14.915508 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-a1e194bc-4175-4b70-978b-6f553c17d15f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9ebd05b5afc42028af1064478e9f5fc', 'request_id': 'req-a1e194bc-4175-4b70-978b-6f553c17d15f'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:14.927689 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-a1e194bc-4175-4b70-978b-6f553c17d15f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9ebd05b5afc42028af1064478e9f5fc', 'request_id': 'req-a1e194bc-4175-4b70-978b-6f553c17d15f'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:14.942653 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-a1e194bc-4175-4b70-978b-6f553c17d15f admin admin] The token cache is being invalidate because project d9ebd05b5afc42028af1064478e9f5fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:52:14.943041 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-a1e194bc-4175-4b70-978b-6f553c17d15f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:14.943602 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-a1e194bc-4175-4b70-978b-6f553c17d15f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:14.945287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 903/1820] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:52:14 2022] DELETE /identity/v3/projects/d9ebd05b5afc42028af1064478e9f5fc => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:14.950308 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a2115-104d-4e90-ae1e-7e97edbd3def None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:14.950764 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a2115-104d-4e90-ae1e-7e97edbd3def None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:14.950840 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a2115-104d-4e90-ae1e-7e97edbd3def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:14.982447 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-246a2115-104d-4e90-ae1e-7e97edbd3def None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:14.983866 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-246a2115-104d-4e90-ae1e-7e97edbd3def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:15.066786 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 918/1821] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:52:15.074781 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3f2cd-d846-43b8-85ae-6cb205e81fc8 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:15.075071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3f2cd-d846-43b8-85ae-6cb205e81fc8 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:15.075347 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3f2cd-d846-43b8-85ae-6cb205e81fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:15.102553 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-46b3f2cd-d846-43b8-85ae-6cb205e81fc8 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:15.102553 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-46b3f2cd-d846-43b8-85ae-6cb205e81fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:15.118945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 904/1822] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:52:15.122990 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a2115-104d-4e90-ae1e-7e97edbd3def None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:15.137232 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a2115-104d-4e90-ae1e-7e97edbd3def None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:15.147284 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15824c8b-8781-40a6-82f4-e18753967828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:15.148236 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15824c8b-8781-40a6-82f4-e18753967828 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:15.148700 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15824c8b-8781-40a6-82f4-e18753967828 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:15.149297 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15824c8b-8781-40a6-82f4-e18753967828 admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:15.150283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15824c8b-8781-40a6-82f4-e18753967828 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:52:15.150759 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15824c8b-8781-40a6-82f4-e18753967828 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:15.152619 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15824c8b-8781-40a6-82f4-e18753967828 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:15.163417 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:15.172640 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 919/1823] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:52:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:52:15.176764 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3f2cd-d846-43b8-85ae-6cb205e81fc8 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:15.186599 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3f2cd-d846-43b8-85ae-6cb205e81fc8 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:15.192181 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf642f-f609-4124-a2dd-a52c3abc880d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:15.193102 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf642f-f609-4124-a2dd-a52c3abc880d admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:15.193530 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf642f-f609-4124-a2dd-a52c3abc880d admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:15.194074 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf642f-f609-4124-a2dd-a52c3abc880d admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:15.195022 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf642f-f609-4124-a2dd-a52c3abc880d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:15.197880 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf642f-f609-4124-a2dd-a52c3abc880d admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:15.319532 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 905/1824] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:52:15 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:52:15.323366 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15824c8b-8781-40a6-82f4-e18753967828 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:15.335403 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15824c8b-8781-40a6-82f4-e18753967828 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:15.403468 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c15e3-4b56-4eca-ab2c-b81dec06c8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:15.404167 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c15e3-4b56-4eca-ab2c-b81dec06c8a2 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:15.404699 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c15e3-4b56-4eca-ab2c-b81dec06c8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:15.404699 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991c15e3-4b56-4eca-ab2c-b81dec06c8a2 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:15.405354 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991c15e3-4b56-4eca-ab2c-b81dec06c8a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:15.407286 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991c15e3-4b56-4eca-ab2c-b81dec06c8a2 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:15.513902 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 920/1825] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:52:15 2022] POST /identity/v3/users => generated 288 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:52:15.518286 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf642f-f609-4124-a2dd-a52c3abc880d admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:15.530949 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf642f-f609-4124-a2dd-a52c3abc880d admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:15.536636 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85f2ff-0b36-4e9c-a39a-6fbbd399df12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:15.537450 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85f2ff-0b36-4e9c-a39a-6fbbd399df12 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:15.537804 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85f2ff-0b36-4e9c-a39a-6fbbd399df12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:15.538151 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85f2ff-0b36-4e9c-a39a-6fbbd399df12 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:15.538871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b85f2ff-0b36-4e9c-a39a-6fbbd399df12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:15.541043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b85f2ff-0b36-4e9c-a39a-6fbbd399df12 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:15.554427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 906/1826] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:52:15.559501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c15e3-4b56-4eca-ab2c-b81dec06c8a2 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:15.573726 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991c15e3-4b56-4eca-ab2c-b81dec06c8a2 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:15.580338 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6668d103-8393-4689-bc9c-e3eb78977a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:15.581873 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668d103-8393-4689-bc9c-e3eb78977a7b admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:15.582248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668d103-8393-4689-bc9c-e3eb78977a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:15.582618 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6668d103-8393-4689-bc9c-e3eb78977a7b admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:15.583393 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6668d103-8393-4689-bc9c-e3eb78977a7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:15.586933 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6668d103-8393-4689-bc9c-e3eb78977a7b admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:15.599956 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 921/1827] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:52:15.604186 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85f2ff-0b36-4e9c-a39a-6fbbd399df12 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:15.614796 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85f2ff-0b36-4e9c-a39a-6fbbd399df12 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:15.619680 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4c35d-b7ea-4fdb-a7a9-9ad3f42f3450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:15.620457 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4c35d-b7ea-4fdb-a7a9-9ad3f42f3450 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:15.620797 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4c35d-b7ea-4fdb-a7a9-9ad3f42f3450 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:15.621077 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4c35d-b7ea-4fdb-a7a9-9ad3f42f3450 admin admin] PATH_INFO: `/v3/projects/7b14e282d3c042e6a347218f8ee900d3/users/66d48cdd5a3b475eaa02b0464d406ab1/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:15.637128 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4c35d-b7ea-4fdb-a7a9-9ad3f42f3450 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b14e282d3c042e6a347218f8ee900d3, user_id=66d48cdd5a3b475eaa02b0464d406ab1, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:15.640140 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4c35d-b7ea-4fdb-a7a9-9ad3f42f3450 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:15.687231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 907/1828] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:52:15 2022] PUT /identity/v3/projects/7b14e282d3c042e6a347218f8ee900d3/users/66d48cdd5a3b475eaa02b0464d406ab1/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:15.697972 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1814bea-620c-4186-9e09-71a56c5d8bab None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:15.698464 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1814bea-620c-4186-9e09-71a56c5d8bab None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:15.698924 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1814bea-620c-4186-9e09-71a56c5d8bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:15.728309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-d1814bea-620c-4186-9e09-71a56c5d8bab None None] MFA Rules not processed for user `66d48cdd5a3b475eaa02b0464d406ab1`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:15.729495 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-d1814bea-620c-4186-9e09-71a56c5d8bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:15.792453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 922/1829] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:15.796927 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4c35d-b7ea-4fdb-a7a9-9ad3f42f3450 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:15.857945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4c35d-b7ea-4fdb-a7a9-9ad3f42f3450 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:15.863144 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28216c4a-aad8-401e-af9a-371d993de08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:15.863762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28216c4a-aad8-401e-af9a-371d993de08d admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:15.864064 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28216c4a-aad8-401e-af9a-371d993de08d admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:15.864338 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28216c4a-aad8-401e-af9a-371d993de08d admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:15.864979 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28216c4a-aad8-401e-af9a-371d993de08d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:15.866863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28216c4a-aad8-401e-af9a-371d993de08d admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:15.893437 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb17bc-7274-4770-9b56-eb0f753dad8c None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:15.893724 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb17bc-7274-4770-9b56-eb0f753dad8c None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:15.893997 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcb17bc-7274-4770-9b56-eb0f753dad8c None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:15.895342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 923/1830] 10.222.0.17 () {54 vars in 921 bytes} [Mon Oct 17 03:52:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 17 03:52:15.905083 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7272f2-4a02-4cf8-8f33-dae11eeecd9f None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:15.905416 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7272f2-4a02-4cf8-8f33-dae11eeecd9f None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:15.905696 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7272f2-4a02-4cf8-8f33-dae11eeecd9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:15.933753 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-fb7272f2-4a02-4cf8-8f33-dae11eeecd9f None None] MFA Rules not processed for user `b500a113a2aa4a95875d7e094b917db3`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:15.935139 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-fb7272f2-4a02-4cf8-8f33-dae11eeecd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:15.981338 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 908/1831] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:52:15 2022] POST /identity/v3/projects => generated 377 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:52:15.983650 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28216c4a-aad8-401e-af9a-371d993de08d admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:16.019898 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 924/1832] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Oct 17 03:52:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:52:16.027723 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:16.031477 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7272f2-4a02-4cf8-8f33-dae11eeecd9f None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:16.033332 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-fb7272f2-4a02-4cf8-8f33-dae11eeecd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:16.042875 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:16.047680 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7272f2-4a02-4cf8-8f33-dae11eeecd9f None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:16.067790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28216c4a-aad8-401e-af9a-371d993de08d admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:16.073377 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b9c04-28ea-4424-84c7-2e270df11c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:16.074262 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b9c04-28ea-4424-84c7-2e270df11c99 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:16.077946 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b9c04-28ea-4424-84c7-2e270df11c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:16.077946 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611b9c04-28ea-4424-84c7-2e270df11c99 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:16.077946 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b9c04-28ea-4424-84c7-2e270df11c99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:16.081780 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611b9c04-28ea-4424-84c7-2e270df11c99 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:16.118018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590f442-837d-4c81-8155-5f77bc89ddf7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b500a113a2aa4a95875d7e094b917db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:16.118676 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590f442-837d-4c81-8155-5f77bc89ddf7 service placement] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:16.118960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590f442-837d-4c81-8155-5f77bc89ddf7 service placement] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:16.119233 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590f442-837d-4c81-8155-5f77bc89ddf7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:16.128499 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7590f442-837d-4c81-8155-5f77bc89ddf7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:16.130081 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7590f442-837d-4c81-8155-5f77bc89ddf7 service placement] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:16.139967 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 925/1833] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:52:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:52:16.140778 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590f442-837d-4c81-8155-5f77bc89ddf7 service placement] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:16.142065 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-7590f442-837d-4c81-8155-5f77bc89ddf7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:16.160966 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7590f442-837d-4c81-8155-5f77bc89ddf7 service placement] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:16.165937 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b961fa9-032e-415a-9b97-9582044f0c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:16.166901 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b961fa9-032e-415a-9b97-9582044f0c96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:16.167322 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b961fa9-032e-415a-9b97-9582044f0c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:16.168096 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b961fa9-032e-415a-9b97-9582044f0c96 admin admin] PATH_INFO: `/v3/users/6307c4e28ee044dab595913514501537` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:16.169327 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b961fa9-032e-415a-9b97-9582044f0c96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6307c4e28ee044dab595913514501537)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:16.171559 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b961fa9-032e-415a-9b97-9582044f0c96 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:16.195579 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 909/1834] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:52:15 2022] POST /identity/v3/users => generated 289 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:52:16.203614 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b9c04-28ea-4424-84c7-2e270df11c99 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:16.214125 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611b9c04-28ea-4424-84c7-2e270df11c99 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:16.218886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b3ba7-b84e-4279-b108-fb6f366607ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:16.219463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b3ba7-b84e-4279-b108-fb6f366607ac admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:16.219761 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b3ba7-b84e-4279-b108-fb6f366607ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:16.220040 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b3ba7-b84e-4279-b108-fb6f366607ac admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:16.220674 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b3ba7-b84e-4279-b108-fb6f366607ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:16.222262 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b3ba7-b84e-4279-b108-fb6f366607ac admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:16.231905 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 910/1835] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:52:16.235440 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b3ba7-b84e-4279-b108-fb6f366607ac admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:16.244702 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b3ba7-b84e-4279-b108-fb6f366607ac admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:16.249219 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04360669-f56b-4109-abcb-97ffe07339c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:16.249783 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04360669-f56b-4109-abcb-97ffe07339c9 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:16.250065 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04360669-f56b-4109-abcb-97ffe07339c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:16.250357 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04360669-f56b-4109-abcb-97ffe07339c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:16.250948 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04360669-f56b-4109-abcb-97ffe07339c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:16.253032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04360669-f56b-4109-abcb-97ffe07339c9 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:16.262603 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 911/1836] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:52:16.266249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04360669-f56b-4109-abcb-97ffe07339c9 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:16.276125 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04360669-f56b-4109-abcb-97ffe07339c9 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:16.280678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7a379-515a-4130-9643-a39610f2e960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:16.281272 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7a379-515a-4130-9643-a39610f2e960 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:16.281555 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7a379-515a-4130-9643-a39610f2e960 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:16.281826 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7a379-515a-4130-9643-a39610f2e960 admin admin] PATH_INFO: `/v3/projects/e331ee400f5d45008de0a378b32974af/users/ae42ab276920464ab7f994586687686d/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:16.296907 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad7a379-515a-4130-9643-a39610f2e960 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e331ee400f5d45008de0a378b32974af, user_id=ae42ab276920464ab7f994586687686d, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:16.299539 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad7a379-515a-4130-9643-a39610f2e960 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:16.412003 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 912/1837] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:52:16 2022] PUT /identity/v3/projects/e331ee400f5d45008de0a378b32974af/users/ae42ab276920464ab7f994586687686d/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:52:16.423533 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5637c4-1bc4-4edb-b8a1-2e17890fc5e7 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:16.423959 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5637c4-1bc4-4edb-b8a1-2e17890fc5e7 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:16.424824 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5637c4-1bc4-4edb-b8a1-2e17890fc5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:16.433481 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-0b961fa9-032e-415a-9b97-9582044f0c96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6307c4e28ee044dab595913514501537'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:16.434124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-0b961fa9-032e-415a-9b97-9582044f0c96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6307c4e28ee044dab595913514501537'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:16.453424 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-1d5637c4-1bc4-4edb-b8a1-2e17890fc5e7 None None] MFA Rules not processed for user `ae42ab276920464ab7f994586687686d`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:16.454617 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-1d5637c4-1bc4-4edb-b8a1-2e17890fc5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:16.517244 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 913/1838] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:52:16.521241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-0b961fa9-032e-415a-9b97-9582044f0c96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6307c4e28ee044dab595913514501537'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:16.521875 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5637c4-1bc4-4edb-b8a1-2e17890fc5e7 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:16.523802 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-1d5637c4-1bc4-4edb-b8a1-2e17890fc5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:16.535666 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-0b961fa9-032e-415a-9b97-9582044f0c96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6307c4e28ee044dab595913514501537'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:16.536407 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-0b961fa9-032e-415a-9b97-9582044f0c96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6307c4e28ee044dab595913514501537'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:16.548789 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 926/1839] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:52:16 2022] DELETE /identity/v3/users/6307c4e28ee044dab595913514501537 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:16.590736 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5637c4-1bc4-4edb-b8a1-2e17890fc5e7 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:16.648955 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a2902-57ff-4a19-8e6f-a193aa4fe22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:16.649612 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a2902-57ff-4a19-8e6f-a193aa4fe22d admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:16.650160 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a2902-57ff-4a19-8e6f-a193aa4fe22d admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:16.650736 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a2902-57ff-4a19-8e6f-a193aa4fe22d admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:16.653206 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:16.653716 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a2902-57ff-4a19-8e6f-a193aa4fe22d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:16.655774 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a2902-57ff-4a19-8e6f-a193aa4fe22d admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:16.813143 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 914/1840] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:52:16 2022] POST /identity/v3/projects => generated 375 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:52:16.817164 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b961fa9-032e-415a-9b97-9582044f0c96 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:16.827363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b961fa9-032e-415a-9b97-9582044f0c96 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:16.842470 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a2902-57ff-4a19-8e6f-a193aa4fe22d admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:16.844499 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-490a2902-57ff-4a19-8e6f-a193aa4fe22d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:16.901265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd1115-6299-451f-9ac0-c757888b102a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:16.901806 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd1115-6299-451f-9ac0-c757888b102a admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:16.902090 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd1115-6299-451f-9ac0-c757888b102a admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:16.902362 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd1115-6299-451f-9ac0-c757888b102a admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:16.903042 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd1115-6299-451f-9ac0-c757888b102a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:16.905039 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd1115-6299-451f-9ac0-c757888b102a admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:16.919333 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:16.930412 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a2902-57ff-4a19-8e6f-a193aa4fe22d admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:16.937546 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715014b2-bc9a-437e-a469-03335530d687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:16.938203 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715014b2-bc9a-437e-a469-03335530d687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:16.938481 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715014b2-bc9a-437e-a469-03335530d687 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:16.938747 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715014b2-bc9a-437e-a469-03335530d687 admin admin] PATH_INFO: `/v3/projects/38bd8dbff7d74bf4baa7c2aeca32c557` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:16.941229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715014b2-bc9a-437e-a469-03335530d687 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38bd8dbff7d74bf4baa7c2aeca32c557)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:16.941875 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715014b2-bc9a-437e-a469-03335530d687 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:17.007783 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 927/1841] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:52:16 2022] POST /identity/v3/users => generated 294 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:52:17.011085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd1115-6299-451f-9ac0-c757888b102a admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:17.023159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd1115-6299-451f-9ac0-c757888b102a admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:17.029015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fb25b-9d2b-4234-9ef1-45b3019d486b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:17.029722 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fb25b-9d2b-4234-9ef1-45b3019d486b admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:17.032476 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fb25b-9d2b-4234-9ef1-45b3019d486b admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:17.032476 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fb25b-9d2b-4234-9ef1-45b3019d486b admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:17.032476 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fb25b-9d2b-4234-9ef1-45b3019d486b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:17.033185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fb25b-9d2b-4234-9ef1-45b3019d486b admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:17.067667 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 928/1842] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:52:17 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:52:17.067879 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-715014b2-bc9a-437e-a469-03335530d687 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38bd8dbff7d74bf4baa7c2aeca32c557', 'request_id': 'req-715014b2-bc9a-437e-a469-03335530d687'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:17.070669 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fb25b-9d2b-4234-9ef1-45b3019d486b admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:17.084947 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fb25b-9d2b-4234-9ef1-45b3019d486b admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:17.137921 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-715014b2-bc9a-437e-a469-03335530d687 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38bd8dbff7d74bf4baa7c2aeca32c557', 'request_id': 'req-715014b2-bc9a-437e-a469-03335530d687'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:17.145575 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-715014b2-bc9a-437e-a469-03335530d687 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38bd8dbff7d74bf4baa7c2aeca32c557', 'request_id': 'req-715014b2-bc9a-437e-a469-03335530d687'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:17.156906 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038f9f1-04bd-44c8-a19e-87c0f9d82bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:17.158043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038f9f1-04bd-44c8-a19e-87c0f9d82bf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:17.158525 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038f9f1-04bd-44c8-a19e-87c0f9d82bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:17.158664 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0038f9f1-04bd-44c8-a19e-87c0f9d82bf8 admin admin] PATH_INFO: `/v3/projects/212364b2c36143079c5dd13d11e879f7/users/2b89774f44bc42dfaa30cdf6507e499c/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:17.170211 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-715014b2-bc9a-437e-a469-03335530d687 admin admin] The token cache is being invalidate because project 38bd8dbff7d74bf4baa7c2aeca32c557 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:52:17.170521 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-715014b2-bc9a-437e-a469-03335530d687 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:17.171392 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-715014b2-bc9a-437e-a469-03335530d687 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:17.173095 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 915/1843] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:52:16 2022] DELETE /identity/v3/projects/38bd8dbff7d74bf4baa7c2aeca32c557 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:17.175232 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0038f9f1-04bd-44c8-a19e-87c0f9d82bf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=212364b2c36143079c5dd13d11e879f7, user_id=2b89774f44bc42dfaa30cdf6507e499c, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:17.178036 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80c59c-1e88-4507-872b-a884d24409f7 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:17.178200 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0038f9f1-04bd-44c8-a19e-87c0f9d82bf8 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:17.178375 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80c59c-1e88-4507-872b-a884d24409f7 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:17.178712 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80c59c-1e88-4507-872b-a884d24409f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:17.198750 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-4d80c59c-1e88-4507-872b-a884d24409f7 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:17.199983 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-4d80c59c-1e88-4507-872b-a884d24409f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:17.208952 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 916/1844] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:52:17.216269 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388052c-43cb-4b6f-b48d-06a2689cae8d None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:17.216556 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388052c-43cb-4b6f-b48d-06a2689cae8d None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:17.216843 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388052c-43cb-4b6f-b48d-06a2689cae8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:17.237348 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-c388052c-43cb-4b6f-b48d-06a2689cae8d None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:17.238600 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c388052c-43cb-4b6f-b48d-06a2689cae8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:17.246774 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 917/1845] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:17.251044 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388052c-43cb-4b6f-b48d-06a2689cae8d None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:17.253922 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 929/1846] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:52:17 2022] PUT /identity/v3/projects/212364b2c36143079c5dd13d11e879f7/users/2b89774f44bc42dfaa30cdf6507e499c/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:52:17.257773 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038f9f1-04bd-44c8-a19e-87c0f9d82bf8 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:17.259571 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-0038f9f1-04bd-44c8-a19e-87c0f9d82bf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:17.262444 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388052c-43cb-4b6f-b48d-06a2689cae8d None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:17.327598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0038f9f1-04bd-44c8-a19e-87c0f9d82bf8 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:17.333711 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095df96d-9a4e-413e-8309-aab29fad53b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:17.334397 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095df96d-9a4e-413e-8309-aab29fad53b5 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:17.334535 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a19ec2-efe2-4fbb-b75b-a81aeca7e845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:17.334833 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095df96d-9a4e-413e-8309-aab29fad53b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:17.335108 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095df96d-9a4e-413e-8309-aab29fad53b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:17.335217 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a19ec2-efe2-4fbb-b75b-a81aeca7e845 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:17.335448 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a19ec2-efe2-4fbb-b75b-a81aeca7e845 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:17.335873 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a19ec2-efe2-4fbb-b75b-a81aeca7e845 admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:17.339758 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095df96d-9a4e-413e-8309-aab29fad53b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:17.339876 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a19ec2-efe2-4fbb-b75b-a81aeca7e845 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:52:17.339876 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a19ec2-efe2-4fbb-b75b-a81aeca7e845 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:17.340009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095df96d-9a4e-413e-8309-aab29fad53b5 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:17.340080 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a19ec2-efe2-4fbb-b75b-a81aeca7e845 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:17.351626 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 918/1847] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:52:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:52:17.355912 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a19ec2-efe2-4fbb-b75b-a81aeca7e845 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:17.361242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 930/1848] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:52:17 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:52:17.365779 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095df96d-9a4e-413e-8309-aab29fad53b5 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:17.374525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a19ec2-efe2-4fbb-b75b-a81aeca7e845 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:17.376110 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095df96d-9a4e-413e-8309-aab29fad53b5 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:17.381322 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d481b894-7601-44a6-9960-ab1068d421e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:17.381996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481b894-7601-44a6-9960-ab1068d421e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:17.382280 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481b894-7601-44a6-9960-ab1068d421e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:17.382567 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481b894-7601-44a6-9960-ab1068d421e8 admin admin] PATH_INFO: `/v3/projects/212364b2c36143079c5dd13d11e879f7/users/2b89774f44bc42dfaa30cdf6507e499c/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:17.384513 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d481b894-7601-44a6-9960-ab1068d421e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=212364b2c36143079c5dd13d11e879f7, user_id=2b89774f44bc42dfaa30cdf6507e499c, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:17.386080 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1cfa88-d7b8-4b38-9828-18a5220f65cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:17.387055 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1cfa88-d7b8-4b38-9828-18a5220f65cb admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:17.387310 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d481b894-7601-44a6-9960-ab1068d421e8 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:17.387718 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1cfa88-d7b8-4b38-9828-18a5220f65cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:17.388346 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1cfa88-d7b8-4b38-9828-18a5220f65cb admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:17.389186 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1cfa88-d7b8-4b38-9828-18a5220f65cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:17.391391 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1cfa88-d7b8-4b38-9828-18a5220f65cb admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:17.399767 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 931/1849] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:52:17 2022] PUT /identity/v3/projects/212364b2c36143079c5dd13d11e879f7/users/2b89774f44bc42dfaa30cdf6507e499c/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:17.401843 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:17.410956 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91204f-0575-4ff2-be3e-4edf29c40556 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:17.411248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91204f-0575-4ff2-be3e-4edf29c40556 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:17.411525 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91204f-0575-4ff2-be3e-4edf29c40556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:17.444928 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-6b91204f-0575-4ff2-be3e-4edf29c40556 None None] MFA Rules not processed for user `2b89774f44bc42dfaa30cdf6507e499c`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:17.446317 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6b91204f-0575-4ff2-be3e-4edf29c40556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:17.527789 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 932/1850] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:52:17.539634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91204f-0575-4ff2-be3e-4edf29c40556 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:17.541316 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6b91204f-0575-4ff2-be3e-4edf29c40556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:17.545248 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 919/1851] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:52:17 2022] POST /identity/v3/projects => generated 383 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:52:17.549116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1cfa88-d7b8-4b38-9828-18a5220f65cb admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:17.599687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91204f-0575-4ff2-be3e-4edf29c40556 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:17.613205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1cfa88-d7b8-4b38-9828-18a5220f65cb admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:17.618185 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaef1c1-22c4-42a3-ae10-7576e0f06fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:17.618791 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaef1c1-22c4-42a3-ae10-7576e0f06fee admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:17.619068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaef1c1-22c4-42a3-ae10-7576e0f06fee admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:17.619341 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaef1c1-22c4-42a3-ae10-7576e0f06fee admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:17.620112 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaef1c1-22c4-42a3-ae10-7576e0f06fee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:17.622030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaef1c1-22c4-42a3-ae10-7576e0f06fee admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:17.635819 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:17.650360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b770e20-216e-49f2-afb5-5e63c1f53f21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:17.650980 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b770e20-216e-49f2-afb5-5e63c1f53f21 service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:17.651261 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b770e20-216e-49f2-afb5-5e63c1f53f21 service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:17.651568 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b770e20-216e-49f2-afb5-5e63c1f53f21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:17.653399 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-3b770e20-216e-49f2-afb5-5e63c1f53f21 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:17.712399 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b770e20-216e-49f2-afb5-5e63c1f53f21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:17.713869 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b770e20-216e-49f2-afb5-5e63c1f53f21 service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:17.718521 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 920/1852] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:52:17 2022] POST /identity/v3/users => generated 292 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:52:17.719487 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaef1c1-22c4-42a3-ae10-7576e0f06fee admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:17.723694 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 933/1853] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:52:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:52:17.730840 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaef1c1-22c4-42a3-ae10-7576e0f06fee admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:17.735521 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7eb86-b376-46c9-b150-d8d8232294ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:17.736167 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7eb86-b376-46c9-b150-d8d8232294ce admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:17.736742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7eb86-b376-46c9-b150-d8d8232294ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:17.737032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7eb86-b376-46c9-b150-d8d8232294ce admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:17.738285 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7eb86-b376-46c9-b150-d8d8232294ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:17.739861 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7eb86-b376-46c9-b150-d8d8232294ce admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:17.750558 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 921/1854] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:52:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:52:17.754825 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b770e20-216e-49f2-afb5-5e63c1f53f21 service cinder] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:17.765606 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b770e20-216e-49f2-afb5-5e63c1f53f21 service cinder] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:17.770922 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3536e-14c2-43e6-93cf-6ef3b92df055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:17.771769 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3536e-14c2-43e6-93cf-6ef3b92df055 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:17.772059 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3536e-14c2-43e6-93cf-6ef3b92df055 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:17.772333 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3536e-14c2-43e6-93cf-6ef3b92df055 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:17.772996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d3536e-14c2-43e6-93cf-6ef3b92df055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:17.774621 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d3536e-14c2-43e6-93cf-6ef3b92df055 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:17.788097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 934/1855] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:52:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:52:17.789361 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7eb86-b376-46c9-b150-d8d8232294ce admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:17.799571 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7eb86-b376-46c9-b150-d8d8232294ce admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:17.804885 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792fd53-9f91-4c78-b010-befa8cef880b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:17.805546 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792fd53-9f91-4c78-b010-befa8cef880b admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:17.805832 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792fd53-9f91-4c78-b010-befa8cef880b admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:17.806112 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792fd53-9f91-4c78-b010-befa8cef880b admin admin] PATH_INFO: `/v3/projects/a139d546eb6a4ff595a94d0e1528ee5d/users/d1499b45cce347c39855da14d66e9690/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:17.822218 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3792fd53-9f91-4c78-b010-befa8cef880b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a139d546eb6a4ff595a94d0e1528ee5d, user_id=d1499b45cce347c39855da14d66e9690, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:17.824897 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3792fd53-9f91-4c78-b010-befa8cef880b admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:17.937864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 922/1856] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:52:17 2022] PUT /identity/v3/projects/a139d546eb6a4ff595a94d0e1528ee5d/users/d1499b45cce347c39855da14d66e9690/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:17.942826 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809afb8e-72b1-4978-b57e-22ffeaffa73c None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:17.943119 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809afb8e-72b1-4978-b57e-22ffeaffa73c None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:17.943403 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809afb8e-72b1-4978-b57e-22ffeaffa73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:17.963105 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-809afb8e-72b1-4978-b57e-22ffeaffa73c None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:17.964409 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-809afb8e-72b1-4978-b57e-22ffeaffa73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:17.999964 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:18.030437 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 935/1857] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:18.040880 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792fd53-9f91-4c78-b010-befa8cef880b admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:18.042900 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-3792fd53-9f91-4c78-b010-befa8cef880b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:18.078268 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:18.101855 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792fd53-9f91-4c78-b010-befa8cef880b admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:18.105655 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93817fb-c6e4-441a-a5d2-f19f8bbbed82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:18.106261 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93817fb-c6e4-441a-a5d2-f19f8bbbed82 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:18.106544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93817fb-c6e4-441a-a5d2-f19f8bbbed82 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:18.106817 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93817fb-c6e4-441a-a5d2-f19f8bbbed82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:18.116767 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93817fb-c6e4-441a-a5d2-f19f8bbbed82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:18.118202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93817fb-c6e4-441a-a5d2-f19f8bbbed82 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:18.127864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 923/1858] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:52:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:52:18.578985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809afb8e-72b1-4978-b57e-22ffeaffa73c None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:18.582089 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-809afb8e-72b1-4978-b57e-22ffeaffa73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:18.594810 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809afb8e-72b1-4978-b57e-22ffeaffa73c None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:18.600002 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e670d-7913-47d7-8940-72f10341c50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:18.600710 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e670d-7913-47d7-8940-72f10341c50c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:18.600981 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e670d-7913-47d7-8940-72f10341c50c admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:18.601313 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e670d-7913-47d7-8940-72f10341c50c admin admin] PATH_INFO: `/v3/users/c9b914e23e8c48d5b7c5a323e14d4caf` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:18.602457 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193e670d-7913-47d7-8940-72f10341c50c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9b914e23e8c48d5b7c5a323e14d4caf)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:18.605427 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193e670d-7913-47d7-8940-72f10341c50c admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:18.898802 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-193e670d-7913-47d7-8940-72f10341c50c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9b914e23e8c48d5b7c5a323e14d4caf'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:18.899540 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-193e670d-7913-47d7-8940-72f10341c50c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9b914e23e8c48d5b7c5a323e14d4caf'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:18.977132 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-193e670d-7913-47d7-8940-72f10341c50c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9b914e23e8c48d5b7c5a323e14d4caf'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:18.990564 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-193e670d-7913-47d7-8940-72f10341c50c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9b914e23e8c48d5b7c5a323e14d4caf'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:18.991226 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-193e670d-7913-47d7-8940-72f10341c50c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9b914e23e8c48d5b7c5a323e14d4caf'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:19.004130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 936/1859] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:52:18 2022] DELETE /identity/v3/users/c9b914e23e8c48d5b7c5a323e14d4caf => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:19.008721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187f1ef-2158-4787-8f6d-b562413c5a1e None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:19.009199 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187f1ef-2158-4787-8f6d-b562413c5a1e None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:19.009317 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d187f1ef-2158-4787-8f6d-b562413c5a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:19.028104 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-d187f1ef-2158-4787-8f6d-b562413c5a1e None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:19.029237 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-d187f1ef-2158-4787-8f6d-b562413c5a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:19.089439 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 924/1860] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:19.099473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e670d-7913-47d7-8940-72f10341c50c admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:19.101000 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-193e670d-7913-47d7-8940-72f10341c50c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:19.160375 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e670d-7913-47d7-8940-72f10341c50c admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:19.164411 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f6e23-0c82-4b26-bd85-8326d251dcb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:19.165267 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f6e23-0c82-4b26-bd85-8326d251dcb9 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:19.165708 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f6e23-0c82-4b26-bd85-8326d251dcb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:19.165986 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8f6e23-0c82-4b26-bd85-8326d251dcb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:19.176042 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f6e23-0c82-4b26-bd85-8326d251dcb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:19.177528 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8f6e23-0c82-4b26-bd85-8326d251dcb9 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:19.189827 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 937/1861] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:52:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:52:19.653041 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187f1ef-2158-4787-8f6d-b562413c5a1e None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:19.654465 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-d187f1ef-2158-4787-8f6d-b562413c5a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:19.680686 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d187f1ef-2158-4787-8f6d-b562413c5a1e None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:19.686887 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582eebfd-eeb0-42e2-ba4e-418fab6c3188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:19.687319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582eebfd-eeb0-42e2-ba4e-418fab6c3188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:19.687596 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582eebfd-eeb0-42e2-ba4e-418fab6c3188 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:19.688082 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582eebfd-eeb0-42e2-ba4e-418fab6c3188 admin admin] PATH_INFO: `/v3/projects/98c4ff300d0840669463225fdf9c9748` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:19.689249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582eebfd-eeb0-42e2-ba4e-418fab6c3188 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98c4ff300d0840669463225fdf9c9748)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:19.692446 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582eebfd-eeb0-42e2-ba4e-418fab6c3188 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:19.911182 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-582eebfd-eeb0-42e2-ba4e-418fab6c3188 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98c4ff300d0840669463225fdf9c9748', 'request_id': 'req-582eebfd-eeb0-42e2-ba4e-418fab6c3188'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:19.944645 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f6e23-0c82-4b26-bd85-8326d251dcb9 service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:19.946086 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-ee8f6e23-0c82-4b26-bd85-8326d251dcb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:20.009795 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8f6e23-0c82-4b26-bd85-8326d251dcb9 service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:20.014290 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:20.015669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-582eebfd-eeb0-42e2-ba4e-418fab6c3188 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98c4ff300d0840669463225fdf9c9748', 'request_id': 'req-582eebfd-eeb0-42e2-ba4e-418fab6c3188'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:20.033684 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-582eebfd-eeb0-42e2-ba4e-418fab6c3188 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98c4ff300d0840669463225fdf9c9748', 'request_id': 'req-582eebfd-eeb0-42e2-ba4e-418fab6c3188'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:20.052440 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-582eebfd-eeb0-42e2-ba4e-418fab6c3188 admin admin] The token cache is being invalidate because project 98c4ff300d0840669463225fdf9c9748 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:52:20.052966 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-582eebfd-eeb0-42e2-ba4e-418fab6c3188 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:20.054648 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-582eebfd-eeb0-42e2-ba4e-418fab6c3188 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:20.061813 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 925/1862] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:52:19 2022] DELETE /identity/v3/projects/98c4ff300d0840669463225fdf9c9748 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:20.062485 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582eebfd-eeb0-42e2-ba4e-418fab6c3188 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:20.064170 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-582eebfd-eeb0-42e2-ba4e-418fab6c3188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:20.085760 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d8702-be8f-4759-8c6b-f46674d50e7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:20.089051 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d8702-be8f-4759-8c6b-f46674d50e7e service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:20.089347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d8702-be8f-4759-8c6b-f46674d50e7e service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:20.089622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6d8702-be8f-4759-8c6b-f46674d50e7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:20.093379 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-8a6d8702-be8f-4759-8c6b-f46674d50e7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:20.179546 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d8702-be8f-4759-8c6b-f46674d50e7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:20.182077 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6d8702-be8f-4759-8c6b-f46674d50e7e service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:20.186535 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582eebfd-eeb0-42e2-ba4e-418fab6c3188 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:20.194361 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7762d8a1-5e5a-41ec-a808-b54c9e2c7c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:20.199278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762d8a1-5e5a-41ec-a808-b54c9e2c7c15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:20.199278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762d8a1-5e5a-41ec-a808-b54c9e2c7c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:20.199278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762d8a1-5e5a-41ec-a808-b54c9e2c7c15 admin admin] PATH_INFO: `/v3/users/5b1f8f7681bd43b3b79f6b91a4c4b8c2` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:20.199278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7762d8a1-5e5a-41ec-a808-b54c9e2c7c15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b1f8f7681bd43b3b79f6b91a4c4b8c2)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:20.200626 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7762d8a1-5e5a-41ec-a808-b54c9e2c7c15 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:20.202554 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:20.212477 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 938/1863] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:52:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:52:20.390757 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-7762d8a1-5e5a-41ec-a808-b54c9e2c7c15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b1f8f7681bd43b3b79f6b91a4c4b8c2'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:20.507677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-7762d8a1-5e5a-41ec-a808-b54c9e2c7c15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b1f8f7681bd43b3b79f6b91a4c4b8c2'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:20.522893 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-7762d8a1-5e5a-41ec-a808-b54c9e2c7c15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b1f8f7681bd43b3b79f6b91a4c4b8c2'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:20.523646 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-7762d8a1-5e5a-41ec-a808-b54c9e2c7c15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b1f8f7681bd43b3b79f6b91a4c4b8c2'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:20.524403 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-7762d8a1-5e5a-41ec-a808-b54c9e2c7c15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b1f8f7681bd43b3b79f6b91a4c4b8c2'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:20.538877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 926/1864] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:52:20 2022] DELETE /identity/v3/users/5b1f8f7681bd43b3b79f6b91a4c4b8c2 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:21.031377 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d8702-be8f-4759-8c6b-f46674d50e7e service cinder] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:21.046522 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6d8702-be8f-4759-8c6b-f46674d50e7e service cinder] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:21.143500 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcdb420-6f75-4107-8aee-8e7bff28b195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:21.144805 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdb420-6f75-4107-8aee-8e7bff28b195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:21.145153 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdb420-6f75-4107-8aee-8e7bff28b195 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:21.145639 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdb420-6f75-4107-8aee-8e7bff28b195 admin admin] PATH_INFO: `/v3/projects/d2c32efd670142f7b42b0162c7038751` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:21.147056 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcdb420-6f75-4107-8aee-8e7bff28b195 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2c32efd670142f7b42b0162c7038751)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:21.149419 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcdb420-6f75-4107-8aee-8e7bff28b195 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:21.418133 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-8fcdb420-6f75-4107-8aee-8e7bff28b195 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2c32efd670142f7b42b0162c7038751', 'request_id': 'req-8fcdb420-6f75-4107-8aee-8e7bff28b195'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:21.507517 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-8fcdb420-6f75-4107-8aee-8e7bff28b195 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2c32efd670142f7b42b0162c7038751', 'request_id': 'req-8fcdb420-6f75-4107-8aee-8e7bff28b195'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:21.536050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-8fcdb420-6f75-4107-8aee-8e7bff28b195 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2c32efd670142f7b42b0162c7038751', 'request_id': 'req-8fcdb420-6f75-4107-8aee-8e7bff28b195'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:21.563613 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-8fcdb420-6f75-4107-8aee-8e7bff28b195 admin admin] The token cache is being invalidate because project d2c32efd670142f7b42b0162c7038751 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:52:21.563948 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-8fcdb420-6f75-4107-8aee-8e7bff28b195 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:21.564919 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-8fcdb420-6f75-4107-8aee-8e7bff28b195 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:21.566675 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 939/1865] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:52:21 2022] DELETE /identity/v3/projects/d2c32efd670142f7b42b0162c7038751 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:21.570759 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7762d8a1-5e5a-41ec-a808-b54c9e2c7c15 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:21.572277 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-7762d8a1-5e5a-41ec-a808-b54c9e2c7c15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:21.586782 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7762d8a1-5e5a-41ec-a808-b54c9e2c7c15 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:21.652677 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:21.668933 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fbf36-41f8-47d9-b20a-5ccb856b86d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:21.668933 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fbf36-41f8-47d9-b20a-5ccb856b86d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:21.668933 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fbf36-41f8-47d9-b20a-5ccb856b86d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:21.668933 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fbf36-41f8-47d9-b20a-5ccb856b86d9 admin admin] PATH_INFO: `/v3/users/433d86187dbd4cfd88e5cd3536bf5c85` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:21.669907 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680fbf36-41f8-47d9-b20a-5ccb856b86d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=433d86187dbd4cfd88e5cd3536bf5c85)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:21.672224 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680fbf36-41f8-47d9-b20a-5ccb856b86d9 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:21.919853 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-680fbf36-41f8-47d9-b20a-5ccb856b86d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '433d86187dbd4cfd88e5cd3536bf5c85'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:22.018508 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-680fbf36-41f8-47d9-b20a-5ccb856b86d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '433d86187dbd4cfd88e5cd3536bf5c85'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:22.034073 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-680fbf36-41f8-47d9-b20a-5ccb856b86d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '433d86187dbd4cfd88e5cd3536bf5c85'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:22.036318 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-680fbf36-41f8-47d9-b20a-5ccb856b86d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '433d86187dbd4cfd88e5cd3536bf5c85'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:22.037083 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-680fbf36-41f8-47d9-b20a-5ccb856b86d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '433d86187dbd4cfd88e5cd3536bf5c85'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:22.054256 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 927/1866] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:52:21 2022] DELETE /identity/v3/users/433d86187dbd4cfd88e5cd3536bf5c85 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:22.068004 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcdb420-6f75-4107-8aee-8e7bff28b195 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:22.069513 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-8fcdb420-6f75-4107-8aee-8e7bff28b195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:22.136823 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcdb420-6f75-4107-8aee-8e7bff28b195 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:22.139073 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:22.200825 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652f92e4-7cea-4779-ae0e-2fe9db09ec02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:22.201872 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f92e4-7cea-4779-ae0e-2fe9db09ec02 service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:22.202463 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f92e4-7cea-4779-ae0e-2fe9db09ec02 service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:22.202849 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652f92e4-7cea-4779-ae0e-2fe9db09ec02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:22.205854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-652f92e4-7cea-4779-ae0e-2fe9db09ec02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:22.287299 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f92e4-7cea-4779-ae0e-2fe9db09ec02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:22.288770 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652f92e4-7cea-4779-ae0e-2fe9db09ec02 service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:22.298772 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 940/1867] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:52:22 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:52:22.577652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fbf36-41f8-47d9-b20a-5ccb856b86d9 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:22.580383 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-680fbf36-41f8-47d9-b20a-5ccb856b86d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:22.739691 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fbf36-41f8-47d9-b20a-5ccb856b86d9 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:22.745711 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d56fb-0f34-444d-ae13-be898f3f8bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:22.746594 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d56fb-0f34-444d-ae13-be898f3f8bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:22.747160 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d56fb-0f34-444d-ae13-be898f3f8bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:22.748706 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d56fb-0f34-444d-ae13-be898f3f8bc7 admin admin] PATH_INFO: `/v3/projects/cda4c94a481945b9b19aaf41135a3bcd` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:22.750032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d56fb-0f34-444d-ae13-be898f3f8bc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cda4c94a481945b9b19aaf41135a3bcd)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:22.752659 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d56fb-0f34-444d-ae13-be898f3f8bc7 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:22.898384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-772d56fb-0f34-444d-ae13-be898f3f8bc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cda4c94a481945b9b19aaf41135a3bcd', 'request_id': 'req-772d56fb-0f34-444d-ae13-be898f3f8bc7'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:22.984606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-772d56fb-0f34-444d-ae13-be898f3f8bc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cda4c94a481945b9b19aaf41135a3bcd', 'request_id': 'req-772d56fb-0f34-444d-ae13-be898f3f8bc7'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:22.992846 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-772d56fb-0f34-444d-ae13-be898f3f8bc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cda4c94a481945b9b19aaf41135a3bcd', 'request_id': 'req-772d56fb-0f34-444d-ae13-be898f3f8bc7'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:23.009828 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-772d56fb-0f34-444d-ae13-be898f3f8bc7 admin admin] The token cache is being invalidate because project cda4c94a481945b9b19aaf41135a3bcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:52:23.010441 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-772d56fb-0f34-444d-ae13-be898f3f8bc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:23.010801 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-772d56fb-0f34-444d-ae13-be898f3f8bc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:23.015656 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 928/1868] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:52:22 2022] DELETE /identity/v3/projects/cda4c94a481945b9b19aaf41135a3bcd => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:23.017549 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625ebee-b2f5-4fa3-9537-5cdeb7022358 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:23.018388 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625ebee-b2f5-4fa3-9537-5cdeb7022358 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:23.019370 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625ebee-b2f5-4fa3-9537-5cdeb7022358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:23.048727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-b625ebee-b2f5-4fa3-9537-5cdeb7022358 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:23.050213 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-b625ebee-b2f5-4fa3-9537-5cdeb7022358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:23.082435 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d56fb-0f34-444d-ae13-be898f3f8bc7 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:23.084118 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-772d56fb-0f34-444d-ae13-be898f3f8bc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:23.163255 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 941/1869] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:52:23.171456 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b259d65-d84a-4975-a842-d60b2464fa0b None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:23.171768 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b259d65-d84a-4975-a842-d60b2464fa0b None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:23.172052 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b259d65-d84a-4975-a842-d60b2464fa0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:23.212920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-6b259d65-d84a-4975-a842-d60b2464fa0b None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:23.214311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6b259d65-d84a-4975-a842-d60b2464fa0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:23.224103 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 942/1870] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:52:23.228708 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b259d65-d84a-4975-a842-d60b2464fa0b None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:23.239158 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b259d65-d84a-4975-a842-d60b2464fa0b None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:23.244509 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c27052-34c9-4b16-9a85-223acd0f5cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:23.246048 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c27052-34c9-4b16-9a85-223acd0f5cc5 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:23.246497 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c27052-34c9-4b16-9a85-223acd0f5cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:23.247032 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c27052-34c9-4b16-9a85-223acd0f5cc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:23.248431 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c27052-34c9-4b16-9a85-223acd0f5cc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:52:23.248745 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c27052-34c9-4b16-9a85-223acd0f5cc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:23.252560 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c27052-34c9-4b16-9a85-223acd0f5cc5 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:23.252702 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d56fb-0f34-444d-ae13-be898f3f8bc7 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:23.259055 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563c95e9-efec-410b-9cf1-34d6e5f5db3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:23.259711 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c95e9-efec-410b-9cf1-34d6e5f5db3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:23.261498 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c95e9-efec-410b-9cf1-34d6e5f5db3a admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:23.261790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563c95e9-efec-410b-9cf1-34d6e5f5db3a admin admin] PATH_INFO: `/v3/users/8cc6b901c1e847458491fd85815efc8b` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:23.263109 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c95e9-efec-410b-9cf1-34d6e5f5db3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cc6b901c1e847458491fd85815efc8b)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:23.265996 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563c95e9-efec-410b-9cf1-34d6e5f5db3a admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:23.276920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 943/1871] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:52:23.276920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c27052-34c9-4b16-9a85-223acd0f5cc5 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:23.286636 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c27052-34c9-4b16-9a85-223acd0f5cc5 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:23.294655 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a3caf7-31d5-40a1-8ce6-af21b8b15d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:23.295844 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3caf7-31d5-40a1-8ce6-af21b8b15d6d admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:23.296440 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3caf7-31d5-40a1-8ce6-af21b8b15d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:23.297023 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3caf7-31d5-40a1-8ce6-af21b8b15d6d admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:23.298358 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a3caf7-31d5-40a1-8ce6-af21b8b15d6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:23.302213 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a3caf7-31d5-40a1-8ce6-af21b8b15d6d admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:23.435369 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 944/1872] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:52:23 2022] POST /identity/v3/projects => generated 379 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:52:23.436578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a3caf7-31d5-40a1-8ce6-af21b8b15d6d admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:23.517070 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-563c95e9-efec-410b-9cf1-34d6e5f5db3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cc6b901c1e847458491fd85815efc8b'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:23.539028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a3caf7-31d5-40a1-8ce6-af21b8b15d6d admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:23.561534 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d14ede-d8bf-4ba3-9136-3a8051358379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:23.562610 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d14ede-d8bf-4ba3-9136-3a8051358379 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:23.563020 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d14ede-d8bf-4ba3-9136-3a8051358379 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:23.563389 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d14ede-d8bf-4ba3-9136-3a8051358379 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:23.564179 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d14ede-d8bf-4ba3-9136-3a8051358379 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:23.566965 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d14ede-d8bf-4ba3-9136-3a8051358379 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:23.610189 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-563c95e9-efec-410b-9cf1-34d6e5f5db3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cc6b901c1e847458491fd85815efc8b'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:23.625130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-563c95e9-efec-410b-9cf1-34d6e5f5db3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cc6b901c1e847458491fd85815efc8b'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:23.626488 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-563c95e9-efec-410b-9cf1-34d6e5f5db3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cc6b901c1e847458491fd85815efc8b'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:23.627268 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-563c95e9-efec-410b-9cf1-34d6e5f5db3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cc6b901c1e847458491fd85815efc8b'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:23.644537 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 929/1873] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:52:23 2022] DELETE /identity/v3/users/8cc6b901c1e847458491fd85815efc8b => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:52:23.649363 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1d642-db20-4d25-bf75-eed4e46e8c32 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:23.649660 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1d642-db20-4d25-bf75-eed4e46e8c32 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:23.649938 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1d642-db20-4d25-bf75-eed4e46e8c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:23.667839 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 945/1874] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:52:23 2022] POST /identity/v3/users => generated 290 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:52:23.668761 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d14ede-d8bf-4ba3-9136-3a8051358379 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:23.670469 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-61d14ede-d8bf-4ba3-9136-3a8051358379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:23.673461 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-76a1d642-db20-4d25-bf75-eed4e46e8c32 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:23.675433 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-76a1d642-db20-4d25-bf75-eed4e46e8c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:23.685557 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d14ede-d8bf-4ba3-9136-3a8051358379 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:23.776050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b11aa17-47ed-4848-93b9-6a462d368aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:23.777526 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11aa17-47ed-4848-93b9-6a462d368aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:23.779158 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11aa17-47ed-4848-93b9-6a462d368aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:23.779158 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b11aa17-47ed-4848-93b9-6a462d368aa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:23.780567 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b11aa17-47ed-4848-93b9-6a462d368aa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:23.782865 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b11aa17-47ed-4848-93b9-6a462d368aa4 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:23.808085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 946/1875] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:52:23.811885 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b11aa17-47ed-4848-93b9-6a462d368aa4 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:23.825479 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b11aa17-47ed-4848-93b9-6a462d368aa4 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:23.827952 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 930/1876] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:23.832158 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789a61d-a0c8-4e73-97ed-75823b85ea64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:23.833024 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789a61d-a0c8-4e73-97ed-75823b85ea64 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:23.833705 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789a61d-a0c8-4e73-97ed-75823b85ea64 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:23.833886 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789a61d-a0c8-4e73-97ed-75823b85ea64 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:23.834674 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d789a61d-a0c8-4e73-97ed-75823b85ea64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:23.834807 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1d642-db20-4d25-bf75-eed4e46e8c32 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:23.836438 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d789a61d-a0c8-4e73-97ed-75823b85ea64 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:23.837327 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-76a1d642-db20-4d25-bf75-eed4e46e8c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:23.849392 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 947/1877] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:52:23.853403 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789a61d-a0c8-4e73-97ed-75823b85ea64 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:23.866547 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789a61d-a0c8-4e73-97ed-75823b85ea64 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:23.873492 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf21130-adc9-4b94-a782-f07f8f44820d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:23.875196 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf21130-adc9-4b94-a782-f07f8f44820d admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:23.875196 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf21130-adc9-4b94-a782-f07f8f44820d admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:23.875196 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf21130-adc9-4b94-a782-f07f8f44820d admin admin] PATH_INFO: `/v3/projects/c561fe5da6e644839d762e7ecef5515c/users/0609cf7282a949b18f3c9d9a9287baeb/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:23.896799 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf21130-adc9-4b94-a782-f07f8f44820d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c561fe5da6e644839d762e7ecef5515c, user_id=0609cf7282a949b18f3c9d9a9287baeb, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:23.899779 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf21130-adc9-4b94-a782-f07f8f44820d admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:23.966754 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a1d642-db20-4d25-bf75-eed4e46e8c32 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:23.972810 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a380ea-8099-4dfe-ae32-47b8685e466d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:23.973599 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a380ea-8099-4dfe-ae32-47b8685e466d service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:23.974427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a380ea-8099-4dfe-ae32-47b8685e466d service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:23.974704 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a380ea-8099-4dfe-ae32-47b8685e466d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:23.986516 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a380ea-8099-4dfe-ae32-47b8685e466d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:23.988204 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a380ea-8099-4dfe-ae32-47b8685e466d service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:23.999275 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 931/1878] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:52:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:52:24.023797 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 948/1879] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:52:23 2022] PUT /identity/v3/projects/c561fe5da6e644839d762e7ecef5515c/users/0609cf7282a949b18f3c9d9a9287baeb/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:24.024767 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb19e0-b65b-44cb-949a-84f55963bbf9 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:24.025085 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb19e0-b65b-44cb-949a-84f55963bbf9 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:24.025361 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb19e0-b65b-44cb-949a-84f55963bbf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:24.050603 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-cbdb19e0-b65b-44cb-949a-84f55963bbf9 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:24.052027 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-cbdb19e0-b65b-44cb-949a-84f55963bbf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:24.139910 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 932/1880] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:24.152597 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf21130-adc9-4b94-a782-f07f8f44820d admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:24.270895 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf21130-adc9-4b94-a782-f07f8f44820d admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:24.274724 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da5ef2e-af1f-4ce6-a8d7-ae4bfe79fc91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:24.275273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5ef2e-af1f-4ce6-a8d7-ae4bfe79fc91 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:24.275553 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5ef2e-af1f-4ce6-a8d7-ae4bfe79fc91 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:24.275908 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da5ef2e-af1f-4ce6-a8d7-ae4bfe79fc91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:24.285884 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da5ef2e-af1f-4ce6-a8d7-ae4bfe79fc91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:24.287471 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da5ef2e-af1f-4ce6-a8d7-ae4bfe79fc91 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:24.309564 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 949/1881] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:52:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:52:24.487560 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:24.922828 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ccc1fe-67ee-4c9c-8054-ecc4f6600496 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:24.923876 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ccc1fe-67ee-4c9c-8054-ecc4f6600496 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:24.925163 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ccc1fe-67ee-4c9c-8054-ecc4f6600496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:24.955403 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-00ccc1fe-67ee-4c9c-8054-ecc4f6600496 None None] MFA Rules not processed for user `d1499b45cce347c39855da14d66e9690`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:24.956657 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-00ccc1fe-67ee-4c9c-8054-ecc4f6600496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:25.015352 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:25.018159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da5ef2e-af1f-4ce6-a8d7-ae4bfe79fc91 service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:25.019674 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-7da5ef2e-af1f-4ce6-a8d7-ae4bfe79fc91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:25.033795 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da5ef2e-af1f-4ce6-a8d7-ae4bfe79fc91 service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:25.039804 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d759d2-07f2-4be4-be13-ea23e6e077b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:25.040389 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d759d2-07f2-4be4-be13-ea23e6e077b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:25.040670 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d759d2-07f2-4be4-be13-ea23e6e077b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:25.041108 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d759d2-07f2-4be4-be13-ea23e6e077b3 admin admin] PATH_INFO: `/v3/projects/7bc690e6c888404fbfc488c898ad2f17` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:25.042261 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d759d2-07f2-4be4-be13-ea23e6e077b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bc690e6c888404fbfc488c898ad2f17)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:25.045040 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d759d2-07f2-4be4-be13-ea23e6e077b3 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:25.050300 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 933/1882] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:25.065765 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ccc1fe-67ee-4c9c-8054-ecc4f6600496 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:25.067878 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-00ccc1fe-67ee-4c9c-8054-ecc4f6600496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:25.173782 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ccc1fe-67ee-4c9c-8054-ecc4f6600496 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:25.173782 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa23c0c-ce05-441e-9f58-b40afb022d78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:25.173782 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa23c0c-ce05-441e-9f58-b40afb022d78 service nova] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:25.173782 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa23c0c-ce05-441e-9f58-b40afb022d78 service nova] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:25.174137 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa23c0c-ce05-441e-9f58-b40afb022d78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:25.174137 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa23c0c-ce05-441e-9f58-b40afb022d78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:25.174137 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa23c0c-ce05-441e-9f58-b40afb022d78 service nova] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:25.182633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 934/1883] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:52:25 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:52:25.340263 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-55d759d2-07f2-4be4-be13-ea23e6e077b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bc690e6c888404fbfc488c898ad2f17', 'request_id': 'req-55d759d2-07f2-4be4-be13-ea23e6e077b3'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:25.438656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-55d759d2-07f2-4be4-be13-ea23e6e077b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bc690e6c888404fbfc488c898ad2f17', 'request_id': 'req-55d759d2-07f2-4be4-be13-ea23e6e077b3'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:25.445422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-55d759d2-07f2-4be4-be13-ea23e6e077b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bc690e6c888404fbfc488c898ad2f17', 'request_id': 'req-55d759d2-07f2-4be4-be13-ea23e6e077b3'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:25.470235 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-55d759d2-07f2-4be4-be13-ea23e6e077b3 admin admin] The token cache is being invalidate because project 7bc690e6c888404fbfc488c898ad2f17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:52:25.470525 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-55d759d2-07f2-4be4-be13-ea23e6e077b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:25.474912 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-55d759d2-07f2-4be4-be13-ea23e6e077b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:25.475312 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 950/1884] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:52:25 2022] DELETE /identity/v3/projects/7bc690e6c888404fbfc488c898ad2f17 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:52:25.480226 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc221a85-2653-4401-9ba8-22ba07e9424b None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:25.480515 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc221a85-2653-4401-9ba8-22ba07e9424b None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:25.480792 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc221a85-2653-4401-9ba8-22ba07e9424b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:25.503047 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-cc221a85-2653-4401-9ba8-22ba07e9424b None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:25.504421 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-cc221a85-2653-4401-9ba8-22ba07e9424b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:25.571184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:25.587115 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 935/1885] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:25.597357 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30de1d-c14d-4085-860a-4ce614d472e4 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:25.597595 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30de1d-c14d-4085-860a-4ce614d472e4 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:25.597872 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30de1d-c14d-4085-860a-4ce614d472e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:25.639123 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-ed30de1d-c14d-4085-860a-4ce614d472e4 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:25.640540 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-ed30de1d-c14d-4085-860a-4ce614d472e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:25.650432 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 951/1886] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:25.654708 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc221a85-2653-4401-9ba8-22ba07e9424b None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:25.665106 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc221a85-2653-4401-9ba8-22ba07e9424b None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:25.670773 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b8d91-3056-431e-8e14-9bd4dfae5402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:25.671416 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b8d91-3056-431e-8e14-9bd4dfae5402 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:25.674222 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b8d91-3056-431e-8e14-9bd4dfae5402 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:25.674222 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74b8d91-3056-431e-8e14-9bd4dfae5402 admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:25.674222 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b8d91-3056-431e-8e14-9bd4dfae5402 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:52:25.674222 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b8d91-3056-431e-8e14-9bd4dfae5402 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:25.674673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74b8d91-3056-431e-8e14-9bd4dfae5402 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:25.690065 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 936/1887] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:52:25.694728 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30de1d-c14d-4085-860a-4ce614d472e4 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:25.706374 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30de1d-c14d-4085-860a-4ce614d472e4 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:25.712708 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd50601-df50-496c-8718-911755d98317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:25.713332 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd50601-df50-496c-8718-911755d98317 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:25.713618 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd50601-df50-496c-8718-911755d98317 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:25.713893 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd50601-df50-496c-8718-911755d98317 admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:25.714545 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd50601-df50-496c-8718-911755d98317 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:25.716546 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd50601-df50-496c-8718-911755d98317 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:25.817163 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 952/1888] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:52:25 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:52:25.818690 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b8d91-3056-431e-8e14-9bd4dfae5402 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:25.830428 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74b8d91-3056-431e-8e14-9bd4dfae5402 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:25.927178 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24864a42-35f1-4a82-869c-d31ba97edbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:25.927880 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24864a42-35f1-4a82-869c-d31ba97edbd3 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:25.928105 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24864a42-35f1-4a82-869c-d31ba97edbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:25.928377 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24864a42-35f1-4a82-869c-d31ba97edbd3 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:25.929014 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24864a42-35f1-4a82-869c-d31ba97edbd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:25.931111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24864a42-35f1-4a82-869c-d31ba97edbd3 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:26.012325 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 937/1889] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:52:25 2022] POST /identity/v3/users => generated 287 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:52:26.016409 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd50601-df50-496c-8718-911755d98317 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:26.027890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd50601-df50-496c-8718-911755d98317 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:26.032486 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfa154-248f-469c-9a58-61a27e25febd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:26.033146 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfa154-248f-469c-9a58-61a27e25febd admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:26.033423 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfa154-248f-469c-9a58-61a27e25febd admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:26.033743 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfa154-248f-469c-9a58-61a27e25febd admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:26.034616 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfa154-248f-469c-9a58-61a27e25febd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:26.036293 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfa154-248f-469c-9a58-61a27e25febd admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:26.049385 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 953/1890] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:52:26.050739 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24864a42-35f1-4a82-869c-d31ba97edbd3 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:26.060889 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24864a42-35f1-4a82-869c-d31ba97edbd3 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:26.066123 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2869bb5-1d6f-46c4-abf7-bc1358a22085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:26.066803 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2869bb5-1d6f-46c4-abf7-bc1358a22085 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:26.067271 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2869bb5-1d6f-46c4-abf7-bc1358a22085 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:26.067629 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2869bb5-1d6f-46c4-abf7-bc1358a22085 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:26.068307 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2869bb5-1d6f-46c4-abf7-bc1358a22085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:26.069917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2869bb5-1d6f-46c4-abf7-bc1358a22085 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:26.082732 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 938/1891] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:52:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:52:26.087047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfa154-248f-469c-9a58-61a27e25febd admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:26.110429 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfa154-248f-469c-9a58-61a27e25febd admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:26.119501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509391e-6beb-4336-843f-1c1ac7855d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:26.120386 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509391e-6beb-4336-843f-1c1ac7855d48 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:26.121416 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509391e-6beb-4336-843f-1c1ac7855d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:26.121800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509391e-6beb-4336-843f-1c1ac7855d48 admin admin] PATH_INFO: `/v3/projects/25d7abaaf76a4147afb1acb1ed2385b2/users/1200fc635ffa41caa5e46cacaa66b5aa/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:26.156662 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509391e-6beb-4336-843f-1c1ac7855d48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25d7abaaf76a4147afb1acb1ed2385b2, user_id=1200fc635ffa41caa5e46cacaa66b5aa, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:26.159709 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509391e-6beb-4336-843f-1c1ac7855d48 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:26.184850 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2ef8f-eaee-45d1-9627-11fff49afe7d None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:26.185142 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2ef8f-eaee-45d1-9627-11fff49afe7d None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:26.185418 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2ef8f-eaee-45d1-9627-11fff49afe7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:26.224160 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-2ff2ef8f-eaee-45d1-9627-11fff49afe7d None None] MFA Rules not processed for user `0609cf7282a949b18f3c9d9a9287baeb`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:26.225147 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-2ff2ef8f-eaee-45d1-9627-11fff49afe7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:26.267822 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 954/1892] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:52:26 2022] PUT /identity/v3/projects/25d7abaaf76a4147afb1acb1ed2385b2/users/1200fc635ffa41caa5e46cacaa66b5aa/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:26.269286 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956534d-e262-4ce6-93af-6fa60854c640 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:26.269575 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956534d-e262-4ce6-93af-6fa60854c640 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:26.269853 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956534d-e262-4ce6-93af-6fa60854c640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:26.309683 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-0956534d-e262-4ce6-93af-6fa60854c640 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:26.311197 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-0956534d-e262-4ce6-93af-6fa60854c640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:26.327062 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 939/1893] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:26.341554 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2ef8f-eaee-45d1-9627-11fff49afe7d None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:26.344143 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-2ff2ef8f-eaee-45d1-9627-11fff49afe7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:26.375611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 955/1894] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:26.386231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956534d-e262-4ce6-93af-6fa60854c640 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:26.388309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-0956534d-e262-4ce6-93af-6fa60854c640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:26.423777 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2ef8f-eaee-45d1-9627-11fff49afe7d None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:26.481800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956534d-e262-4ce6-93af-6fa60854c640 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:26.487719 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4097e7c1-3458-421e-a82d-3e9d68556d92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:26.488365 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097e7c1-3458-421e-a82d-3e9d68556d92 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:26.488798 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097e7c1-3458-421e-a82d-3e9d68556d92 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:26.489802 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4097e7c1-3458-421e-a82d-3e9d68556d92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:26.494560 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf58ce6-d4ca-48c6-bee2-b6bf19dbbebc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:26.495308 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf58ce6-d4ca-48c6-bee2-b6bf19dbbebc service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:26.496306 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf58ce6-d4ca-48c6-bee2-b6bf19dbbebc service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:26.496688 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf58ce6-d4ca-48c6-bee2-b6bf19dbbebc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:26.502047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4097e7c1-3458-421e-a82d-3e9d68556d92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:26.504176 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4097e7c1-3458-421e-a82d-3e9d68556d92 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:26.521626 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 956/1895] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:52:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:52:26.570097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf58ce6-d4ca-48c6-bee2-b6bf19dbbebc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:26.571254 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf58ce6-d4ca-48c6-bee2-b6bf19dbbebc service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:26.581940 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 940/1896] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:52:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:52:27.583531 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:28.045780 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:28.054660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d80c07-39ef-4b57-bbd5-83bbb480dbe5 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:28.055014 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d80c07-39ef-4b57-bbd5-83bbb480dbe5 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:28.055318 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d80c07-39ef-4b57-bbd5-83bbb480dbe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:28.075263 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-d9d80c07-39ef-4b57-bbd5-83bbb480dbe5 None None] MFA Rules not processed for user `d1499b45cce347c39855da14d66e9690`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:28.076517 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-d9d80c07-39ef-4b57-bbd5-83bbb480dbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:28.142265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 957/1897] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:28.280746 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec982f8-3662-4e60-a40c-be44d9f6350a None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:28.281183 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec982f8-3662-4e60-a40c-be44d9f6350a None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:28.282348 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec982f8-3662-4e60-a40c-be44d9f6350a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:28.332094 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-3ec982f8-3662-4e60-a40c-be44d9f6350a None None] MFA Rules not processed for user `1200fc635ffa41caa5e46cacaa66b5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:28.333890 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-3ec982f8-3662-4e60-a40c-be44d9f6350a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:28.462825 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 941/1898] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:28.470668 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d80c07-39ef-4b57-bbd5-83bbb480dbe5 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:28.487886 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d80c07-39ef-4b57-bbd5-83bbb480dbe5 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:28.492601 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba8a77-29fe-46cb-9742-309eabadd208 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:28.493896 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba8a77-29fe-46cb-9742-309eabadd208 service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:28.494378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba8a77-29fe-46cb-9742-309eabadd208 service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:28.494830 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ba8a77-29fe-46cb-9742-309eabadd208 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:28.506974 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ba8a77-29fe-46cb-9742-309eabadd208 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:28.511177 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ba8a77-29fe-46cb-9742-309eabadd208 service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:28.539166 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 958/1899] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:52:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:52:28.580165 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a3df7-210c-41e5-abdf-dd456143ebaf None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:28.580165 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a3df7-210c-41e5-abdf-dd456143ebaf None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:28.583272 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a3df7-210c-41e5-abdf-dd456143ebaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:28.634351 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-487a3df7-210c-41e5-abdf-dd456143ebaf None None] MFA Rules not processed for user `0609cf7282a949b18f3c9d9a9287baeb`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:28.637343 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-487a3df7-210c-41e5-abdf-dd456143ebaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:28.647692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 942/1900] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:29.762244 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba8a77-29fe-46cb-9742-309eabadd208 service cinder] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:29.763662 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-c0ba8a77-29fe-46cb-9742-309eabadd208 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:29.777614 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ba8a77-29fe-46cb-9742-309eabadd208 service cinder] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:29.782390 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368f063-91e8-478a-9a03-8476b6b9b107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:29.784785 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368f063-91e8-478a-9a03-8476b6b9b107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:29.784785 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368f063-91e8-478a-9a03-8476b6b9b107 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:29.784785 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6368f063-91e8-478a-9a03-8476b6b9b107 admin admin] PATH_INFO: `/v3/users/66d48cdd5a3b475eaa02b0464d406ab1` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:29.785261 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6368f063-91e8-478a-9a03-8476b6b9b107 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66d48cdd5a3b475eaa02b0464d406ab1)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:29.787156 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6368f063-91e8-478a-9a03-8476b6b9b107 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:29.989357 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-6368f063-91e8-478a-9a03-8476b6b9b107 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66d48cdd5a3b475eaa02b0464d406ab1'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:29.990131 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-6368f063-91e8-478a-9a03-8476b6b9b107 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66d48cdd5a3b475eaa02b0464d406ab1'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:30.063702 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-6368f063-91e8-478a-9a03-8476b6b9b107 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66d48cdd5a3b475eaa02b0464d406ab1'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:30.091174 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-6368f063-91e8-478a-9a03-8476b6b9b107 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66d48cdd5a3b475eaa02b0464d406ab1'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:30.092005 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-6368f063-91e8-478a-9a03-8476b6b9b107 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66d48cdd5a3b475eaa02b0464d406ab1'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:30.106359 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 959/1901] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:52:29 2022] DELETE /identity/v3/users/66d48cdd5a3b475eaa02b0464d406ab1 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:30.111178 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9d608-9ed2-44bc-b3f9-62a59cc333ab None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:30.112437 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9d608-9ed2-44bc-b3f9-62a59cc333ab None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:30.113221 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a9d608-9ed2-44bc-b3f9-62a59cc333ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:30.132494 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-21a9d608-9ed2-44bc-b3f9-62a59cc333ab None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:30.137419 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-21a9d608-9ed2-44bc-b3f9-62a59cc333ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:30.200912 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 943/1902] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:30.212347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368f063-91e8-478a-9a03-8476b6b9b107 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:30.217483 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6368f063-91e8-478a-9a03-8476b6b9b107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:30.278313 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6368f063-91e8-478a-9a03-8476b6b9b107 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:30.282035 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f46232-6174-4ac1-95e3-bbaede981d53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:30.282661 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f46232-6174-4ac1-95e3-bbaede981d53 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:30.282944 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f46232-6174-4ac1-95e3-bbaede981d53 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:30.283219 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f46232-6174-4ac1-95e3-bbaede981d53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:30.292380 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f46232-6174-4ac1-95e3-bbaede981d53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:30.293879 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f46232-6174-4ac1-95e3-bbaede981d53 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:30.303443 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 960/1903] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:52:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:52:30.370343 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:30.661315 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596157b-0ff6-4a84-9722-3728f7b40808 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:30.662097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596157b-0ff6-4a84-9722-3728f7b40808 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:30.662097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f596157b-0ff6-4a84-9722-3728f7b40808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:30.692149 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-f596157b-0ff6-4a84-9722-3728f7b40808 None None] MFA Rules not processed for user `1200fc635ffa41caa5e46cacaa66b5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:30.693568 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-f596157b-0ff6-4a84-9722-3728f7b40808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:30.721970 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:30.767034 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 944/1904] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:31.051264 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f46232-6174-4ac1-95e3-bbaede981d53 service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:31.055275 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-84f46232-6174-4ac1-95e3-bbaede981d53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:31.086683 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f46232-6174-4ac1-95e3-bbaede981d53 service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:31.102474 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bfd8a8-fcb9-4b96-95c8-9d53556df1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:31.103826 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfd8a8-fcb9-4b96-95c8-9d53556df1bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:31.104281 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfd8a8-fcb9-4b96-95c8-9d53556df1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:31.104710 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bfd8a8-fcb9-4b96-95c8-9d53556df1bf admin admin] PATH_INFO: `/v3/projects/7b14e282d3c042e6a347218f8ee900d3` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:31.107528 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bfd8a8-fcb9-4b96-95c8-9d53556df1bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b14e282d3c042e6a347218f8ee900d3)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:31.109517 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bfd8a8-fcb9-4b96-95c8-9d53556df1bf admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:31.265979 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-16bfd8a8-fcb9-4b96-95c8-9d53556df1bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b14e282d3c042e6a347218f8ee900d3', 'request_id': 'req-16bfd8a8-fcb9-4b96-95c8-9d53556df1bf'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:31.426889 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-16bfd8a8-fcb9-4b96-95c8-9d53556df1bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b14e282d3c042e6a347218f8ee900d3', 'request_id': 'req-16bfd8a8-fcb9-4b96-95c8-9d53556df1bf'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:31.433472 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-16bfd8a8-fcb9-4b96-95c8-9d53556df1bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b14e282d3c042e6a347218f8ee900d3', 'request_id': 'req-16bfd8a8-fcb9-4b96-95c8-9d53556df1bf'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:31.445136 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-16bfd8a8-fcb9-4b96-95c8-9d53556df1bf admin admin] The token cache is being invalidate because project 7b14e282d3c042e6a347218f8ee900d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:52:31.445425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-16bfd8a8-fcb9-4b96-95c8-9d53556df1bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:31.446075 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-16bfd8a8-fcb9-4b96-95c8-9d53556df1bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:31.447574 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 961/1905] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:52:31 2022] DELETE /identity/v3/projects/7b14e282d3c042e6a347218f8ee900d3 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:31.451886 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f596157b-0ff6-4a84-9722-3728f7b40808 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:31.464510 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f596157b-0ff6-4a84-9722-3728f7b40808 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:31.524774 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:31.530581 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b598e1-d478-40f6-8ee0-b75bc94dadba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:31.534273 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b598e1-d478-40f6-8ee0-b75bc94dadba admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:31.534273 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b598e1-d478-40f6-8ee0-b75bc94dadba admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:31.534273 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b598e1-d478-40f6-8ee0-b75bc94dadba admin admin] PATH_INFO: `/v3/users/ae42ab276920464ab7f994586687686d` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:31.534273 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b598e1-d478-40f6-8ee0-b75bc94dadba admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae42ab276920464ab7f994586687686d)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:31.547160 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b598e1-d478-40f6-8ee0-b75bc94dadba admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:31.813833 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-45b598e1-d478-40f6-8ee0-b75bc94dadba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae42ab276920464ab7f994586687686d'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:31.908524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-45b598e1-d478-40f6-8ee0-b75bc94dadba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae42ab276920464ab7f994586687686d'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:31.922426 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-45b598e1-d478-40f6-8ee0-b75bc94dadba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae42ab276920464ab7f994586687686d'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:31.923126 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-45b598e1-d478-40f6-8ee0-b75bc94dadba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae42ab276920464ab7f994586687686d'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:31.923766 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-45b598e1-d478-40f6-8ee0-b75bc94dadba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae42ab276920464ab7f994586687686d'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:31.939415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 945/1906] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:52:31 2022] DELETE /identity/v3/users/ae42ab276920464ab7f994586687686d => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:32.480202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bfd8a8-fcb9-4b96-95c8-9d53556df1bf admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:32.483375 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-16bfd8a8-fcb9-4b96-95c8-9d53556df1bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:32.500349 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bfd8a8-fcb9-4b96-95c8-9d53556df1bf admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:32.588335 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d0243-4880-420b-8509-d2faf9f6d849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:32.589022 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d0243-4880-420b-8509-d2faf9f6d849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:32.589879 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d0243-4880-420b-8509-d2faf9f6d849 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:32.590179 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9d0243-4880-420b-8509-d2faf9f6d849 admin admin] PATH_INFO: `/v3/projects/e331ee400f5d45008de0a378b32974af` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:32.591509 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9d0243-4880-420b-8509-d2faf9f6d849 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e331ee400f5d45008de0a378b32974af)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:32.593664 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9d0243-4880-420b-8509-d2faf9f6d849 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:32.597925 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:32.709163 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2e9d0243-4880-420b-8509-d2faf9f6d849 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e331ee400f5d45008de0a378b32974af', 'request_id': 'req-2e9d0243-4880-420b-8509-d2faf9f6d849'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:32.798105 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2e9d0243-4880-420b-8509-d2faf9f6d849 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e331ee400f5d45008de0a378b32974af', 'request_id': 'req-2e9d0243-4880-420b-8509-d2faf9f6d849'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:32.804947 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2e9d0243-4880-420b-8509-d2faf9f6d849 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e331ee400f5d45008de0a378b32974af', 'request_id': 'req-2e9d0243-4880-420b-8509-d2faf9f6d849'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:32.816127 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2e9d0243-4880-420b-8509-d2faf9f6d849 admin admin] The token cache is being invalidate because project e331ee400f5d45008de0a378b32974af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:52:32.816346 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2e9d0243-4880-420b-8509-d2faf9f6d849 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:32.817151 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2e9d0243-4880-420b-8509-d2faf9f6d849 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:32.818627 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 962/1907] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:52:32 2022] DELETE /identity/v3/projects/e331ee400f5d45008de0a378b32974af => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:32.822852 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b598e1-d478-40f6-8ee0-b75bc94dadba admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:32.824294 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-45b598e1-d478-40f6-8ee0-b75bc94dadba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:32.828527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:32.839692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b598e1-d478-40f6-8ee0-b75bc94dadba admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:32.897510 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c9d29-0b51-43f7-9a42-9a07c5ce1d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:32.898149 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c9d29-0b51-43f7-9a42-9a07c5ce1d0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:32.898431 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c9d29-0b51-43f7-9a42-9a07c5ce1d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:32.898707 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c9d29-0b51-43f7-9a42-9a07c5ce1d0d admin admin] PATH_INFO: `/v3/users/2b89774f44bc42dfaa30cdf6507e499c` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:32.899862 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1c9d29-0b51-43f7-9a42-9a07c5ce1d0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b89774f44bc42dfaa30cdf6507e499c)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:32.901847 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1c9d29-0b51-43f7-9a42-9a07c5ce1d0d admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:33.104929 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-fa1c9d29-0b51-43f7-9a42-9a07c5ce1d0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b89774f44bc42dfaa30cdf6507e499c'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:33.154171 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-fa1c9d29-0b51-43f7-9a42-9a07c5ce1d0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b89774f44bc42dfaa30cdf6507e499c'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:33.167717 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-fa1c9d29-0b51-43f7-9a42-9a07c5ce1d0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b89774f44bc42dfaa30cdf6507e499c'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:33.168389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-fa1c9d29-0b51-43f7-9a42-9a07c5ce1d0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b89774f44bc42dfaa30cdf6507e499c'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:33.168997 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-fa1c9d29-0b51-43f7-9a42-9a07c5ce1d0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b89774f44bc42dfaa30cdf6507e499c'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:33.181371 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 946/1908] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:52:32 2022] DELETE /identity/v3/users/2b89774f44bc42dfaa30cdf6507e499c => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:33.223305 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:33.658239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d0243-4880-420b-8509-d2faf9f6d849 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:33.659848 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-2e9d0243-4880-420b-8509-d2faf9f6d849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:33.680304 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9d0243-4880-420b-8509-d2faf9f6d849 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:33.742269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6d359-06c5-4881-b5a1-d6b8860cc418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:33.743402 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6d359-06c5-4881-b5a1-d6b8860cc418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:33.744147 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6d359-06c5-4881-b5a1-d6b8860cc418 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:33.744631 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6d359-06c5-4881-b5a1-d6b8860cc418 admin admin] PATH_INFO: `/v3/projects/212364b2c36143079c5dd13d11e879f7` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:33.746898 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e6d359-06c5-4881-b5a1-d6b8860cc418 admin admin] RBAC: Authorizing `identity:delete_project(project_id=212364b2c36143079c5dd13d11e879f7)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:33.750009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e6d359-06c5-4881-b5a1-d6b8860cc418 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:33.869571 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-a1e6d359-06c5-4881-b5a1-d6b8860cc418 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '212364b2c36143079c5dd13d11e879f7', 'request_id': 'req-a1e6d359-06c5-4881-b5a1-d6b8860cc418'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:33.930981 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-a1e6d359-06c5-4881-b5a1-d6b8860cc418 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '212364b2c36143079c5dd13d11e879f7', 'request_id': 'req-a1e6d359-06c5-4881-b5a1-d6b8860cc418'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:33.937929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-a1e6d359-06c5-4881-b5a1-d6b8860cc418 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '212364b2c36143079c5dd13d11e879f7', 'request_id': 'req-a1e6d359-06c5-4881-b5a1-d6b8860cc418'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:33.950646 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-a1e6d359-06c5-4881-b5a1-d6b8860cc418 admin admin] The token cache is being invalidate because project 212364b2c36143079c5dd13d11e879f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:52:33.950960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-a1e6d359-06c5-4881-b5a1-d6b8860cc418 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:33.951917 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-a1e6d359-06c5-4881-b5a1-d6b8860cc418 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:33.953417 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 963/1909] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:52:33 2022] DELETE /identity/v3/projects/212364b2c36143079c5dd13d11e879f7 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:33.959200 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1accf-805e-417e-9ce6-16acf100976b None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:33.959686 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1accf-805e-417e-9ce6-16acf100976b None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:33.959818 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a1accf-805e-417e-9ce6-16acf100976b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:33.979943 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-07a1accf-805e-417e-9ce6-16acf100976b None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:33.981136 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-07a1accf-805e-417e-9ce6-16acf100976b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:34.051320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 947/1910] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:34.060010 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148ba32-784b-4280-afd3-c74330dd3cac None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:34.060290 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148ba32-784b-4280-afd3-c74330dd3cac None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:34.060562 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148ba32-784b-4280-afd3-c74330dd3cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:34.081238 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-1148ba32-784b-4280-afd3-c74330dd3cac None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:34.082449 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-1148ba32-784b-4280-afd3-c74330dd3cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:34.090543 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 964/1911] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:34.094936 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1accf-805e-417e-9ce6-16acf100976b None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:34.105175 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a1accf-805e-417e-9ce6-16acf100976b None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:34.110136 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d7687-1c1a-4059-a846-7d5c23091771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:34.112192 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d7687-1c1a-4059-a846-7d5c23091771 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:34.112192 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d7687-1c1a-4059-a846-7d5c23091771 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:34.112192 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d7687-1c1a-4059-a846-7d5c23091771 admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:34.112665 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d7687-1c1a-4059-a846-7d5c23091771 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:52:34.112969 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d7687-1c1a-4059-a846-7d5c23091771 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:34.114544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d7687-1c1a-4059-a846-7d5c23091771 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:34.126549 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 948/1912] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:52:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:52:34.133844 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1148ba32-784b-4280-afd3-c74330dd3cac None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:34.145706 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1148ba32-784b-4280-afd3-c74330dd3cac None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:34.150723 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af5ad9-ec01-4695-81ce-b0ebb5d17f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:34.151327 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af5ad9-ec01-4695-81ce-b0ebb5d17f02 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:34.151605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af5ad9-ec01-4695-81ce-b0ebb5d17f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:34.151934 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af5ad9-ec01-4695-81ce-b0ebb5d17f02 admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:34.152673 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af5ad9-ec01-4695-81ce-b0ebb5d17f02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:34.154547 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af5ad9-ec01-4695-81ce-b0ebb5d17f02 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:34.170424 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:34.270118 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 965/1913] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:52:34 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:52:34.274225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d7687-1c1a-4059-a846-7d5c23091771 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:34.284626 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d7687-1c1a-4059-a846-7d5c23091771 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:34.308831 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:34.340496 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9686010-4230-4219-abe8-087db3abb711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:34.341133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9686010-4230-4219-abe8-087db3abb711 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:34.341414 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9686010-4230-4219-abe8-087db3abb711 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:34.341689 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9686010-4230-4219-abe8-087db3abb711 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:34.342327 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9686010-4230-4219-abe8-087db3abb711 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:34.344251 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9686010-4230-4219-abe8-087db3abb711 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:34.443242 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 949/1914] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:52:34 2022] POST /identity/v3/users => generated 293 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:52:34.447249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af5ad9-ec01-4695-81ce-b0ebb5d17f02 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:34.458233 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af5ad9-ec01-4695-81ce-b0ebb5d17f02 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:34.462767 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79510d5c-6c18-4f73-8802-3c37b349d6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:34.463357 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79510d5c-6c18-4f73-8802-3c37b349d6ad admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:34.463632 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79510d5c-6c18-4f73-8802-3c37b349d6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:34.463937 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79510d5c-6c18-4f73-8802-3c37b349d6ad admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:34.464570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79510d5c-6c18-4f73-8802-3c37b349d6ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:34.466131 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79510d5c-6c18-4f73-8802-3c37b349d6ad admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:34.476468 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 966/1915] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:52:34.480308 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9686010-4230-4219-abe8-087db3abb711 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:34.490142 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9686010-4230-4219-abe8-087db3abb711 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:34.494765 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8f8a3-ab57-4b10-b029-07b19516cdbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:34.495358 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8f8a3-ab57-4b10-b029-07b19516cdbf admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:34.495655 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8f8a3-ab57-4b10-b029-07b19516cdbf admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:34.495962 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8f8a3-ab57-4b10-b029-07b19516cdbf admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:34.496601 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8f8a3-ab57-4b10-b029-07b19516cdbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:34.498166 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8f8a3-ab57-4b10-b029-07b19516cdbf admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:34.507831 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 950/1916] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:52:34.511676 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79510d5c-6c18-4f73-8802-3c37b349d6ad admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:34.521931 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79510d5c-6c18-4f73-8802-3c37b349d6ad admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:34.526487 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125dd6df-c832-4049-8513-41afbcefe23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:34.527097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125dd6df-c832-4049-8513-41afbcefe23e admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:34.527374 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125dd6df-c832-4049-8513-41afbcefe23e admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:34.527645 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125dd6df-c832-4049-8513-41afbcefe23e admin admin] PATH_INFO: `/v3/projects/8430b2e483264d118e6bcc3113cf213a/users/6fb112d0eb104e6d810fa1db8794e3f6/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:34.542269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125dd6df-c832-4049-8513-41afbcefe23e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8430b2e483264d118e6bcc3113cf213a, user_id=6fb112d0eb104e6d810fa1db8794e3f6, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:34.544994 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125dd6df-c832-4049-8513-41afbcefe23e admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:34.611278 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 967/1917] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:52:34 2022] PUT /identity/v3/projects/8430b2e483264d118e6bcc3113cf213a/users/6fb112d0eb104e6d810fa1db8794e3f6/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:34.621660 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548f400-6b3b-413a-95ae-fc65c98b3306 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:34.621950 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548f400-6b3b-413a-95ae-fc65c98b3306 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:34.622224 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548f400-6b3b-413a-95ae-fc65c98b3306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:34.651277 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-8548f400-6b3b-413a-95ae-fc65c98b3306 None None] MFA Rules not processed for user `6fb112d0eb104e6d810fa1db8794e3f6`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:34.652562 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-8548f400-6b3b-413a-95ae-fc65c98b3306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:34.722427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 951/1918] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:34.733378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125dd6df-c832-4049-8513-41afbcefe23e admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:34.735139 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-125dd6df-c832-4049-8513-41afbcefe23e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:34.793798 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125dd6df-c832-4049-8513-41afbcefe23e admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:34.797383 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc023264-5f23-4ced-8297-8739f686da8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:34.798024 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc023264-5f23-4ced-8297-8739f686da8f service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:34.798302 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc023264-5f23-4ced-8297-8739f686da8f service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:34.798570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc023264-5f23-4ced-8297-8739f686da8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:34.806967 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc023264-5f23-4ced-8297-8739f686da8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:34.808413 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc023264-5f23-4ced-8297-8739f686da8f service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:34.817517 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 968/1919] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:52:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:52:38.067030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548f400-6b3b-413a-95ae-fc65c98b3306 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:38.150178 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548f400-6b3b-413a-95ae-fc65c98b3306 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:38.157405 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86646218-542c-4853-9ba3-e8374d47dd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:38.158686 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86646218-542c-4853-9ba3-e8374d47dd11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:38.158686 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86646218-542c-4853-9ba3-e8374d47dd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:38.158894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86646218-542c-4853-9ba3-e8374d47dd11 admin admin] PATH_INFO: `/v3/users/6fb112d0eb104e6d810fa1db8794e3f6` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:38.160562 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86646218-542c-4853-9ba3-e8374d47dd11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fb112d0eb104e6d810fa1db8794e3f6)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:38.162329 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86646218-542c-4853-9ba3-e8374d47dd11 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:38.434869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:38.444556 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-86646218-542c-4853-9ba3-e8374d47dd11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fb112d0eb104e6d810fa1db8794e3f6'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:38.567425 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-86646218-542c-4853-9ba3-e8374d47dd11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fb112d0eb104e6d810fa1db8794e3f6'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:38.588358 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-86646218-542c-4853-9ba3-e8374d47dd11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fb112d0eb104e6d810fa1db8794e3f6'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:38.589335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-86646218-542c-4853-9ba3-e8374d47dd11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fb112d0eb104e6d810fa1db8794e3f6'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:38.591114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-86646218-542c-4853-9ba3-e8374d47dd11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fb112d0eb104e6d810fa1db8794e3f6'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:38.616816 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239922c7-979e-49c2-a5b9-0af40b467f92 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:38.617865 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239922c7-979e-49c2-a5b9-0af40b467f92 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:38.618322 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239922c7-979e-49c2-a5b9-0af40b467f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:38.622117 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 952/1920] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:52:38 2022] DELETE /identity/v3/users/6fb112d0eb104e6d810fa1db8794e3f6 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:38.647129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-239922c7-979e-49c2-a5b9-0af40b467f92 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:38.648890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-239922c7-979e-49c2-a5b9-0af40b467f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:38.734418 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 969/1921] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:38.751387 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86646218-542c-4853-9ba3-e8374d47dd11 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:38.753563 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-86646218-542c-4853-9ba3-e8374d47dd11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:38.823445 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86646218-542c-4853-9ba3-e8374d47dd11 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:38.828310 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d4633a-935a-4923-a77f-681206ef8843 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:38.829022 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4633a-935a-4923-a77f-681206ef8843 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:38.829627 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4633a-935a-4923-a77f-681206ef8843 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:38.830534 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d4633a-935a-4923-a77f-681206ef8843 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:38.851950 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d4633a-935a-4923-a77f-681206ef8843 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:38.851950 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d4633a-935a-4923-a77f-681206ef8843 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:38.856720 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 953/1922] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:52:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:52:39.487305 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239922c7-979e-49c2-a5b9-0af40b467f92 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:39.489061 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-239922c7-979e-49c2-a5b9-0af40b467f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:39.501766 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239922c7-979e-49c2-a5b9-0af40b467f92 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:39.507291 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c74faf-10c7-430a-aa86-675787358e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:39.507978 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c74faf-10c7-430a-aa86-675787358e9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:39.509545 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c74faf-10c7-430a-aa86-675787358e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:39.509545 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c74faf-10c7-430a-aa86-675787358e9f admin admin] PATH_INFO: `/v3/projects/8430b2e483264d118e6bcc3113cf213a` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:39.510460 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c74faf-10c7-430a-aa86-675787358e9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8430b2e483264d118e6bcc3113cf213a)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:39.512546 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c74faf-10c7-430a-aa86-675787358e9f admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:39.655019 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-86c74faf-10c7-430a-aa86-675787358e9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8430b2e483264d118e6bcc3113cf213a', 'request_id': 'req-86c74faf-10c7-430a-aa86-675787358e9f'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:39.737183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-86c74faf-10c7-430a-aa86-675787358e9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8430b2e483264d118e6bcc3113cf213a', 'request_id': 'req-86c74faf-10c7-430a-aa86-675787358e9f'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:39.738572 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:39.743832 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-86c74faf-10c7-430a-aa86-675787358e9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8430b2e483264d118e6bcc3113cf213a', 'request_id': 'req-86c74faf-10c7-430a-aa86-675787358e9f'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:39.755608 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-86c74faf-10c7-430a-aa86-675787358e9f admin admin] The token cache is being invalidate because project 8430b2e483264d118e6bcc3113cf213a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:52:39.755953 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-86c74faf-10c7-430a-aa86-675787358e9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:39.756589 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-86c74faf-10c7-430a-aa86-675787358e9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:39.758077 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 970/1923] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:52:39 2022] DELETE /identity/v3/projects/8430b2e483264d118e6bcc3113cf213a => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:41.606823 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d4633a-935a-4923-a77f-681206ef8843 service neutron] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:41.608246 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-21d4633a-935a-4923-a77f-681206ef8843 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:41.623535 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d4633a-935a-4923-a77f-681206ef8843 service neutron] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:41.685233 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b6771-bfa1-40c9-8335-caf737027087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:41.685871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b6771-bfa1-40c9-8335-caf737027087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:41.686148 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b6771-bfa1-40c9-8335-caf737027087 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:41.686419 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b6771-bfa1-40c9-8335-caf737027087 admin admin] PATH_INFO: `/v3/users/fc753dbadafa493db7b2c2b4b5ad4289` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:41.687525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b6771-bfa1-40c9-8335-caf737027087 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc753dbadafa493db7b2c2b4b5ad4289)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:41.689497 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b6771-bfa1-40c9-8335-caf737027087 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:41.934818 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-839b6771-bfa1-40c9-8335-caf737027087 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc753dbadafa493db7b2c2b4b5ad4289'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:42.014806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-839b6771-bfa1-40c9-8335-caf737027087 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc753dbadafa493db7b2c2b4b5ad4289'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:42.020559 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:42.028635 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-839b6771-bfa1-40c9-8335-caf737027087 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc753dbadafa493db7b2c2b4b5ad4289'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:42.029298 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-839b6771-bfa1-40c9-8335-caf737027087 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc753dbadafa493db7b2c2b4b5ad4289'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:42.030082 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-839b6771-bfa1-40c9-8335-caf737027087 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc753dbadafa493db7b2c2b4b5ad4289'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:42.043048 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 954/1924] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:52:41 2022] DELETE /identity/v3/users/fc753dbadafa493db7b2c2b4b5ad4289 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:42.047534 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327fffd-ac66-4f78-a0b5-2dbe9a248934 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:42.048045 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327fffd-ac66-4f78-a0b5-2dbe9a248934 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:42.048173 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327fffd-ac66-4f78-a0b5-2dbe9a248934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:42.067124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-7327fffd-ac66-4f78-a0b5-2dbe9a248934 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:42.068607 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-7327fffd-ac66-4f78-a0b5-2dbe9a248934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:42.127555 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 971/1925] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:52:42.137500 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b6771-bfa1-40c9-8335-caf737027087 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:42.139213 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-839b6771-bfa1-40c9-8335-caf737027087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:42.199446 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b6771-bfa1-40c9-8335-caf737027087 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:42.203317 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9a85a-f273-4638-a3da-c1847e379b51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:42.207225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9a85a-f273-4638-a3da-c1847e379b51 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:42.207225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9a85a-f273-4638-a3da-c1847e379b51 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:42.207225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9a85a-f273-4638-a3da-c1847e379b51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:42.214735 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f9a85a-f273-4638-a3da-c1847e379b51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:42.216227 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f9a85a-f273-4638-a3da-c1847e379b51 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:42.226213 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 955/1926] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:52:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:52:42.721635 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327fffd-ac66-4f78-a0b5-2dbe9a248934 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:42.723543 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-7327fffd-ac66-4f78-a0b5-2dbe9a248934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:42.744983 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327fffd-ac66-4f78-a0b5-2dbe9a248934 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:42.750107 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0342ba-87ea-4dc9-85ad-f7bf8a1a1215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:42.750790 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0342ba-87ea-4dc9-85ad-f7bf8a1a1215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:42.751065 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0342ba-87ea-4dc9-85ad-f7bf8a1a1215 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:42.751349 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0342ba-87ea-4dc9-85ad-f7bf8a1a1215 admin admin] PATH_INFO: `/v3/projects/226cbf5a2fc74c2f96023776b52f85a5` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:42.753114 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0342ba-87ea-4dc9-85ad-f7bf8a1a1215 admin admin] RBAC: Authorizing `identity:delete_project(project_id=226cbf5a2fc74c2f96023776b52f85a5)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:42.755062 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0342ba-87ea-4dc9-85ad-f7bf8a1a1215 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:42.862593 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-8c0342ba-87ea-4dc9-85ad-f7bf8a1a1215 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '226cbf5a2fc74c2f96023776b52f85a5', 'request_id': 'req-8c0342ba-87ea-4dc9-85ad-f7bf8a1a1215'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:42.925195 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-8c0342ba-87ea-4dc9-85ad-f7bf8a1a1215 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '226cbf5a2fc74c2f96023776b52f85a5', 'request_id': 'req-8c0342ba-87ea-4dc9-85ad-f7bf8a1a1215'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:42.931542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-8c0342ba-87ea-4dc9-85ad-f7bf8a1a1215 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '226cbf5a2fc74c2f96023776b52f85a5', 'request_id': 'req-8c0342ba-87ea-4dc9-85ad-f7bf8a1a1215'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:42.943634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-8c0342ba-87ea-4dc9-85ad-f7bf8a1a1215 admin admin] The token cache is being invalidate because project 226cbf5a2fc74c2f96023776b52f85a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:52:42.943980 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-8c0342ba-87ea-4dc9-85ad-f7bf8a1a1215 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:42.944639 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-8c0342ba-87ea-4dc9-85ad-f7bf8a1a1215 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:52:42.946165 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 972/1927] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:52:42 2022] DELETE /identity/v3/projects/226cbf5a2fc74c2f96023776b52f85a5 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:42.950946 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a01b3-475b-4894-a864-7c4ed4af88f3 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:42.951434 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a01b3-475b-4894-a864-7c4ed4af88f3 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:42.951638 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1a01b3-475b-4894-a864-7c4ed4af88f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:42.971983 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-1a1a01b3-475b-4894-a864-7c4ed4af88f3 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:42.973187 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-1a1a01b3-475b-4894-a864-7c4ed4af88f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:43.031865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 956/1928] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:43.040204 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea542f-2a94-4d0f-8517-8234f24d52e6 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:43.040515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea542f-2a94-4d0f-8517-8234f24d52e6 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:43.040772 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecea542f-2a94-4d0f-8517-8234f24d52e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:43.061287 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-ecea542f-2a94-4d0f-8517-8234f24d52e6 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:43.062615 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-ecea542f-2a94-4d0f-8517-8234f24d52e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:43.070616 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 973/1929] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:43.074761 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a01b3-475b-4894-a864-7c4ed4af88f3 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:43.084770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1a01b3-475b-4894-a864-7c4ed4af88f3 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:43.089558 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a9c54d-e6dd-4f55-90de-a25215801ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:43.090193 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9c54d-e6dd-4f55-90de-a25215801ec1 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:43.090496 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9c54d-e6dd-4f55-90de-a25215801ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:43.090768 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9c54d-e6dd-4f55-90de-a25215801ec1 admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:43.092993 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a9c54d-e6dd-4f55-90de-a25215801ec1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:52:43.092993 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a9c54d-e6dd-4f55-90de-a25215801ec1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:43.093333 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a9c54d-e6dd-4f55-90de-a25215801ec1 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:43.105255 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 957/1930] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:52:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:52:43.109464 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea542f-2a94-4d0f-8517-8234f24d52e6 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:43.119444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecea542f-2a94-4d0f-8517-8234f24d52e6 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:43.124242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd490b-8616-4ffe-a2b9-0952f4f912bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:43.124831 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd490b-8616-4ffe-a2b9-0952f4f912bf admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:43.125110 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd490b-8616-4ffe-a2b9-0952f4f912bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:43.125377 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bd490b-8616-4ffe-a2b9-0952f4f912bf admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:43.126835 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd490b-8616-4ffe-a2b9-0952f4f912bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:43.128895 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bd490b-8616-4ffe-a2b9-0952f4f912bf admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:43.291207 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 974/1931] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:52:43 2022] POST /identity/v3/projects => generated 389 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:52:43.292409 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a9c54d-e6dd-4f55-90de-a25215801ec1 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:43.302943 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a9c54d-e6dd-4f55-90de-a25215801ec1 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:43.361092 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a9d0c-557a-44b9-b5e8-056258c0494a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:43.361710 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a9d0c-557a-44b9-b5e8-056258c0494a admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:43.361992 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a9d0c-557a-44b9-b5e8-056258c0494a admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:43.362290 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a9d0c-557a-44b9-b5e8-056258c0494a admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:43.362943 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a9d0c-557a-44b9-b5e8-056258c0494a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:43.364937 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a9d0c-557a-44b9-b5e8-056258c0494a admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:43.376342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:43.463792 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 958/1932] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:52:43 2022] POST /identity/v3/users => generated 295 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:52:43.467890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd490b-8616-4ffe-a2b9-0952f4f912bf admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:43.480768 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bd490b-8616-4ffe-a2b9-0952f4f912bf admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:43.487222 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf634e-1ccb-4e93-83b4-5021448fb9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:43.487481 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf634e-1ccb-4e93-83b4-5021448fb9f1 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:43.487898 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf634e-1ccb-4e93-83b4-5021448fb9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:43.488271 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf634e-1ccb-4e93-83b4-5021448fb9f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:43.489202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf634e-1ccb-4e93-83b4-5021448fb9f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:43.492696 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf634e-1ccb-4e93-83b4-5021448fb9f1 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:43.509187 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 975/1933] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:52:43.515835 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a9d0c-557a-44b9-b5e8-056258c0494a admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:43.535524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a9d0c-557a-44b9-b5e8-056258c0494a admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:43.548448 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d37a8-e71c-449b-92ed-b2befcb735f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:43.549088 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d37a8-e71c-449b-92ed-b2befcb735f7 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:43.549373 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d37a8-e71c-449b-92ed-b2befcb735f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:43.549647 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d37a8-e71c-449b-92ed-b2befcb735f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:43.550329 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d37a8-e71c-449b-92ed-b2befcb735f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:43.551973 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d37a8-e71c-449b-92ed-b2befcb735f7 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:43.564697 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 959/1934] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:52:43 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:52:43.568758 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf634e-1ccb-4e93-83b4-5021448fb9f1 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:43.580469 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf634e-1ccb-4e93-83b4-5021448fb9f1 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:43.586073 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0bb9c-1b75-41cd-a464-1ee660f29389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:43.586910 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0bb9c-1b75-41cd-a464-1ee660f29389 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:43.587189 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0bb9c-1b75-41cd-a464-1ee660f29389 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:43.587460 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0bb9c-1b75-41cd-a464-1ee660f29389 admin admin] PATH_INFO: `/v3/projects/64d2516411af4714bd54bb2f82c903cc/users/74f4ba94c63b437f9fe25877c4799204/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:43.612886 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f0bb9c-1b75-41cd-a464-1ee660f29389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64d2516411af4714bd54bb2f82c903cc, user_id=74f4ba94c63b437f9fe25877c4799204, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:43.617857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f0bb9c-1b75-41cd-a464-1ee660f29389 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:43.725645 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 976/1935] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:52:43 2022] PUT /identity/v3/projects/64d2516411af4714bd54bb2f82c903cc/users/74f4ba94c63b437f9fe25877c4799204/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:52:43.737101 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9319af2-ed69-4534-87c7-423e4780790c None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:43.737340 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9319af2-ed69-4534-87c7-423e4780790c None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:43.737616 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9319af2-ed69-4534-87c7-423e4780790c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:43.771335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-c9319af2-ed69-4534-87c7-423e4780790c None None] MFA Rules not processed for user `74f4ba94c63b437f9fe25877c4799204`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:52:43.772818 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c9319af2-ed69-4534-87c7-423e4780790c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:43.861294 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 960/1936] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:52:43 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:52:43.868730 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0bb9c-1b75-41cd-a464-1ee660f29389 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:52:43.870507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-52f0bb9c-1b75-41cd-a464-1ee660f29389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:52:43.878006 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:43.955528 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0bb9c-1b75-41cd-a464-1ee660f29389 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:52:43.960261 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f143522-0cf2-4a6d-a50d-007e6e20288a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:52:43.960261 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f143522-0cf2-4a6d-a50d-007e6e20288a service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:52:43.960261 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f143522-0cf2-4a6d-a50d-007e6e20288a service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:52:43.960511 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f143522-0cf2-4a6d-a50d-007e6e20288a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:52:43.969395 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f143522-0cf2-4a6d-a50d-007e6e20288a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:52:43.970865 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f143522-0cf2-4a6d-a50d-007e6e20288a service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:52:43.980753 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 977/1937] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:52:43 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:52:54.130615 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=39 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:52:54.587620 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=24 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:53:06.010489 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9319af2-ed69-4534-87c7-423e4780790c None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:53:06.081322 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9319af2-ed69-4534-87c7-423e4780790c None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:53:06.085947 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a84a3-3d86-44a4-a857-1b63b383aa5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:53:06.086584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a84a3-3d86-44a4-a857-1b63b383aa5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:06.086865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a84a3-3d86-44a4-a857-1b63b383aa5f admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:06.087136 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a84a3-3d86-44a4-a857-1b63b383aa5f admin admin] PATH_INFO: `/v3/users/74f4ba94c63b437f9fe25877c4799204` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:06.088288 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949a84a3-3d86-44a4-a857-1b63b383aa5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=74f4ba94c63b437f9fe25877c4799204)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:53:06.090429 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949a84a3-3d86-44a4-a857-1b63b383aa5f admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:53:06.279441 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-949a84a3-3d86-44a4-a857-1b63b383aa5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74f4ba94c63b437f9fe25877c4799204'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:53:06.358363 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-949a84a3-3d86-44a4-a857-1b63b383aa5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74f4ba94c63b437f9fe25877c4799204'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:53:06.372233 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-949a84a3-3d86-44a4-a857-1b63b383aa5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74f4ba94c63b437f9fe25877c4799204'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:53:06.372976 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-949a84a3-3d86-44a4-a857-1b63b383aa5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74f4ba94c63b437f9fe25877c4799204'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:53:06.378696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-949a84a3-3d86-44a4-a857-1b63b383aa5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74f4ba94c63b437f9fe25877c4799204'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:53:06.387697 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 961/1938] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:53:06 2022] DELETE /identity/v3/users/74f4ba94c63b437f9fe25877c4799204 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:53:06.392292 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a137f-5dba-408a-a1f7-8ab2944bd236 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:06.392763 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a137f-5dba-408a-a1f7-8ab2944bd236 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:06.392846 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a137f-5dba-408a-a1f7-8ab2944bd236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:06.412592 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-1c4a137f-5dba-408a-a1f7-8ab2944bd236 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:53:06.413801 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-1c4a137f-5dba-408a-a1f7-8ab2944bd236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:53:06.473331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 978/1939] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:53:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:53:06.482728 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a84a3-3d86-44a4-a857-1b63b383aa5f admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:53:06.484184 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-949a84a3-3d86-44a4-a857-1b63b383aa5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:53:06.545770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a84a3-3d86-44a4-a857-1b63b383aa5f admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:53:06.545770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9441c9c0-b648-4f08-9a7f-4981397a688b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:53:06.545770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441c9c0-b648-4f08-9a7f-4981397a688b service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:06.545770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441c9c0-b648-4f08-9a7f-4981397a688b service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:06.546364 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441c9c0-b648-4f08-9a7f-4981397a688b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:06.554687 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9441c9c0-b648-4f08-9a7f-4981397a688b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:53:06.556168 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9441c9c0-b648-4f08-9a7f-4981397a688b service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:53:06.565569 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 962/1940] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:53:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:53:07.094637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a137f-5dba-408a-a1f7-8ab2944bd236 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:53:07.096305 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-1c4a137f-5dba-408a-a1f7-8ab2944bd236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:53:07.109243 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a137f-5dba-408a-a1f7-8ab2944bd236 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:53:07.118561 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1957d9-ac3e-4fed-847e-602fa0b9aef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:53:07.119355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1957d9-ac3e-4fed-847e-602fa0b9aef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:07.119635 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1957d9-ac3e-4fed-847e-602fa0b9aef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:07.119990 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1957d9-ac3e-4fed-847e-602fa0b9aef6 admin admin] PATH_INFO: `/v3/projects/64d2516411af4714bd54bb2f82c903cc` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:07.121125 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1957d9-ac3e-4fed-847e-602fa0b9aef6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64d2516411af4714bd54bb2f82c903cc)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:53:07.124199 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1957d9-ac3e-4fed-847e-602fa0b9aef6 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:53:07.211045 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-7e1957d9-ac3e-4fed-847e-602fa0b9aef6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64d2516411af4714bd54bb2f82c903cc', 'request_id': 'req-7e1957d9-ac3e-4fed-847e-602fa0b9aef6'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:53:07.255213 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-7e1957d9-ac3e-4fed-847e-602fa0b9aef6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64d2516411af4714bd54bb2f82c903cc', 'request_id': 'req-7e1957d9-ac3e-4fed-847e-602fa0b9aef6'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:53:07.261482 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-7e1957d9-ac3e-4fed-847e-602fa0b9aef6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64d2516411af4714bd54bb2f82c903cc', 'request_id': 'req-7e1957d9-ac3e-4fed-847e-602fa0b9aef6'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:53:07.273575 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-7e1957d9-ac3e-4fed-847e-602fa0b9aef6 admin admin] The token cache is being invalidate because project 64d2516411af4714bd54bb2f82c903cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:53:07.273865 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-7e1957d9-ac3e-4fed-847e-602fa0b9aef6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:53:07.274503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-7e1957d9-ac3e-4fed-847e-602fa0b9aef6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:53:07.276042 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 979/1941] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:53:07 2022] DELETE /identity/v3/projects/64d2516411af4714bd54bb2f82c903cc => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:53:07.280876 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63240413-f26a-4c25-b996-b1abea62c465 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:07.281355 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63240413-f26a-4c25-b996-b1abea62c465 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:07.281451 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63240413-f26a-4c25-b996-b1abea62c465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:07.300921 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-63240413-f26a-4c25-b996-b1abea62c465 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:53:07.302103 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-63240413-f26a-4c25-b996-b1abea62c465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:53:07.318246 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:53:07.360107 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 963/1942] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:53:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:53:07.368216 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e21c3-70f0-4e6c-aaee-7b889e6f8280 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:07.368502 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e21c3-70f0-4e6c-aaee-7b889e6f8280 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:07.368776 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e21c3-70f0-4e6c-aaee-7b889e6f8280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:07.390049 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-e51e21c3-70f0-4e6c-aaee-7b889e6f8280 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:53:07.391294 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-e51e21c3-70f0-4e6c-aaee-7b889e6f8280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:53:07.399099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 980/1943] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:53:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:53:07.403269 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63240413-f26a-4c25-b996-b1abea62c465 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:53:07.412913 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63240413-f26a-4c25-b996-b1abea62c465 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:53:07.417447 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdada87c-42f5-4343-a8be-fe5e1df72b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:53:07.418044 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdada87c-42f5-4343-a8be-fe5e1df72b13 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:07.418322 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdada87c-42f5-4343-a8be-fe5e1df72b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:07.418593 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdada87c-42f5-4343-a8be-fe5e1df72b13 admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:07.419234 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdada87c-42f5-4343-a8be-fe5e1df72b13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:53:07.419536 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdada87c-42f5-4343-a8be-fe5e1df72b13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:53:07.421149 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdada87c-42f5-4343-a8be-fe5e1df72b13 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:53:07.432827 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 964/1944] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:53:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:53:07.436852 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e21c3-70f0-4e6c-aaee-7b889e6f8280 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:53:07.446417 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e21c3-70f0-4e6c-aaee-7b889e6f8280 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:53:07.450964 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b64878-b651-4584-ba23-ce3de5166e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:53:07.451553 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b64878-b651-4584-ba23-ce3de5166e65 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:07.451866 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b64878-b651-4584-ba23-ce3de5166e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:07.452137 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b64878-b651-4584-ba23-ce3de5166e65 admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:07.452797 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b64878-b651-4584-ba23-ce3de5166e65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:53:07.454917 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b64878-b651-4584-ba23-ce3de5166e65 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:53:07.559338 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 981/1945] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:53:07 2022] POST /identity/v3/projects => generated 397 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:53:07.563148 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdada87c-42f5-4343-a8be-fe5e1df72b13 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:53:07.573744 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdada87c-42f5-4343-a8be-fe5e1df72b13 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:53:07.629592 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c690c4-91f9-452d-9418-74bccbf64318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:53:07.630128 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c690c4-91f9-452d-9418-74bccbf64318 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:07.630412 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c690c4-91f9-452d-9418-74bccbf64318 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:07.630686 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c690c4-91f9-452d-9418-74bccbf64318 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:07.631667 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c690c4-91f9-452d-9418-74bccbf64318 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:53:07.633593 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c690c4-91f9-452d-9418-74bccbf64318 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:53:07.730977 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 965/1946] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:53:07 2022] POST /identity/v3/users => generated 299 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:53:07.734973 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b64878-b651-4584-ba23-ce3de5166e65 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:53:07.745957 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b64878-b651-4584-ba23-ce3de5166e65 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:53:07.750807 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140e5e7-5667-4a1a-8339-eceb0cd31f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:53:07.751434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140e5e7-5667-4a1a-8339-eceb0cd31f99 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:07.751709 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140e5e7-5667-4a1a-8339-eceb0cd31f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:07.752009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8140e5e7-5667-4a1a-8339-eceb0cd31f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:07.752647 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8140e5e7-5667-4a1a-8339-eceb0cd31f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:53:07.754243 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8140e5e7-5667-4a1a-8339-eceb0cd31f99 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:53:07.764959 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 982/1947] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:53:07.768562 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c690c4-91f9-452d-9418-74bccbf64318 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:53:07.778192 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c690c4-91f9-452d-9418-74bccbf64318 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:53:07.782650 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c49491-db02-4b44-9b07-cec7b9c4792a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:53:07.783240 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c49491-db02-4b44-9b07-cec7b9c4792a admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:07.783518 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c49491-db02-4b44-9b07-cec7b9c4792a admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:07.783823 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c49491-db02-4b44-9b07-cec7b9c4792a admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:07.784457 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c49491-db02-4b44-9b07-cec7b9c4792a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:53:07.786502 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c49491-db02-4b44-9b07-cec7b9c4792a admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:53:07.797004 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 966/1948] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:53:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:53:07.801578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140e5e7-5667-4a1a-8339-eceb0cd31f99 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:53:07.811328 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8140e5e7-5667-4a1a-8339-eceb0cd31f99 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:53:07.815766 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1ceee-8fb7-48fb-812a-886e165f301f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:53:07.816583 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1ceee-8fb7-48fb-812a-886e165f301f admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:07.816860 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1ceee-8fb7-48fb-812a-886e165f301f admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:07.817128 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1ceee-8fb7-48fb-812a-886e165f301f admin admin] PATH_INFO: `/v3/projects/3b2d58c8957247b9ae3bfb8439a43983/users/971e84147df7434fae1d2418804fb34c/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:07.831517 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1ceee-8fb7-48fb-812a-886e165f301f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b2d58c8957247b9ae3bfb8439a43983, user_id=971e84147df7434fae1d2418804fb34c, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:53:07.834195 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1ceee-8fb7-48fb-812a-886e165f301f admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:53:07.889747 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 983/1949] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:53:07 2022] PUT /identity/v3/projects/3b2d58c8957247b9ae3bfb8439a43983/users/971e84147df7434fae1d2418804fb34c/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:53:07.901064 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3df313-f1ae-402f-9bbe-ce2983853967 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:07.901064 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3df313-f1ae-402f-9bbe-ce2983853967 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:07.901064 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3df313-f1ae-402f-9bbe-ce2983853967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:07.928787 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-8f3df313-f1ae-402f-9bbe-ce2983853967 None None] MFA Rules not processed for user `971e84147df7434fae1d2418804fb34c`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:53:07.929958 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-8f3df313-f1ae-402f-9bbe-ce2983853967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:53:07.998005 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 967/1950] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:53:07 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:53:08.009801 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1ceee-8fb7-48fb-812a-886e165f301f admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:53:08.011605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-2dd1ceee-8fb7-48fb-812a-886e165f301f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:53:08.041169 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:53:08.080252 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1ceee-8fb7-48fb-812a-886e165f301f admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:53:08.084936 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45555f75-d7f8-4e32-a18e-f7adbd96a708 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:53:08.085555 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45555f75-d7f8-4e32-a18e-f7adbd96a708 service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:08.085854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45555f75-d7f8-4e32-a18e-f7adbd96a708 service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:08.086524 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45555f75-d7f8-4e32-a18e-f7adbd96a708 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:08.098724 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45555f75-d7f8-4e32-a18e-f7adbd96a708 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:53:08.100910 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45555f75-d7f8-4e32-a18e-f7adbd96a708 service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:53:08.111525 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 984/1951] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:53:08 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:53:09.347274 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f0231-6e70-42f4-8273-3024f76b7f9b None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:09.347867 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f0231-6e70-42f4-8273-3024f76b7f9b None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:09.347991 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f0231-6e70-42f4-8273-3024f76b7f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:09.367991 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-4b4f0231-6e70-42f4-8273-3024f76b7f9b None None] MFA Rules not processed for user `1200fc635ffa41caa5e46cacaa66b5aa`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:53:09.369252 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-4b4f0231-6e70-42f4-8273-3024f76b7f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:53:09.382154 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:53:09.425836 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 968/1952] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:53:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:53:17.255894 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45555f75-d7f8-4e32-a18e-f7adbd96a708 service cinder] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:53:17.331430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45555f75-d7f8-4e32-a18e-f7adbd96a708 service cinder] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:53:17.338420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b28b37-02f7-4306-94ee-04170dc2d9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:53:17.339071 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b28b37-02f7-4306-94ee-04170dc2d9e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:17.339354 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b28b37-02f7-4306-94ee-04170dc2d9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:17.339624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b28b37-02f7-4306-94ee-04170dc2d9e0 admin admin] PATH_INFO: `/v3/users/971e84147df7434fae1d2418804fb34c` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:17.341348 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b28b37-02f7-4306-94ee-04170dc2d9e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=971e84147df7434fae1d2418804fb34c)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:53:17.343401 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b28b37-02f7-4306-94ee-04170dc2d9e0 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:53:17.627376 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d1b28b37-02f7-4306-94ee-04170dc2d9e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '971e84147df7434fae1d2418804fb34c'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:53:17.629136 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d1b28b37-02f7-4306-94ee-04170dc2d9e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '971e84147df7434fae1d2418804fb34c'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:53:17.769422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d1b28b37-02f7-4306-94ee-04170dc2d9e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '971e84147df7434fae1d2418804fb34c'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:53:17.818246 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d1b28b37-02f7-4306-94ee-04170dc2d9e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '971e84147df7434fae1d2418804fb34c'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:53:17.820065 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d1b28b37-02f7-4306-94ee-04170dc2d9e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '971e84147df7434fae1d2418804fb34c'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:53:17.845037 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 985/1953] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:53:17 2022] DELETE /identity/v3/users/971e84147df7434fae1d2418804fb34c => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:53:17.851238 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889883c-f64c-41a6-8757-42782143ed43 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:17.852822 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889883c-f64c-41a6-8757-42782143ed43 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:17.853943 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889883c-f64c-41a6-8757-42782143ed43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:17.887385 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-9889883c-f64c-41a6-8757-42782143ed43 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:53:17.888814 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-9889883c-f64c-41a6-8757-42782143ed43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:53:17.975802 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 969/1954] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:53:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:53:17.986462 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b28b37-02f7-4306-94ee-04170dc2d9e0 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:53:17.987982 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-d1b28b37-02f7-4306-94ee-04170dc2d9e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:53:18.050949 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:53:18.064223 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b28b37-02f7-4306-94ee-04170dc2d9e0 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:53:18.069128 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e95bd-cafd-47e0-adbe-f7b666d45f00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:53:18.069361 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e95bd-cafd-47e0-adbe-f7b666d45f00 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:18.069648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e95bd-cafd-47e0-adbe-f7b666d45f00 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:18.069923 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915e95bd-cafd-47e0-adbe-f7b666d45f00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:18.082054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e95bd-cafd-47e0-adbe-f7b666d45f00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:53:18.084824 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915e95bd-cafd-47e0-adbe-f7b666d45f00 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:53:18.098252 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 986/1955] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:53:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:53:18.176291 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a1b23-747c-4c81-9515-b737c575a5a2 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:18.176840 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a1b23-747c-4c81-9515-b737c575a5a2 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:18.177216 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a1b23-747c-4c81-9515-b737c575a5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:18.202012 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-d23a1b23-747c-4c81-9515-b737c575a5a2 None None] MFA Rules not processed for user `cecbc19355ef4570b2b05bba876a3a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:53:18.204992 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-d23a1b23-747c-4c81-9515-b737c575a5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:53:18.283684 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 970/1956] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Oct 17 03:53:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:53:18.330493 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e95bd-cafd-47e0-adbe-f7b666d45f00 service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:53:18.332019 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-915e95bd-cafd-47e0-adbe-f7b666d45f00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:53:18.405198 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915e95bd-cafd-47e0-adbe-f7b666d45f00 service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:53:18.410232 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8356dc16-c382-4acf-9a40-48e37633afa9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f64aaf4d408462a88963973d42abfb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:53:18.410891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356dc16-c382-4acf-9a40-48e37633afa9 service swift] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:18.411175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356dc16-c382-4acf-9a40-48e37633afa9 service swift] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:18.411452 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8356dc16-c382-4acf-9a40-48e37633afa9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:18.421206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8356dc16-c382-4acf-9a40-48e37633afa9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:53:18.422923 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8356dc16-c382-4acf-9a40-48e37633afa9 service swift] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:53:18.432950 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 987/1957] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Oct 17 03:53:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 03:53:18.653319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50422679-489a-4e08-be84-cc42412ae303 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:18.653653 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50422679-489a-4e08-be84-cc42412ae303 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:18.654600 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50422679-489a-4e08-be84-cc42412ae303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:18.681107 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-50422679-489a-4e08-be84-cc42412ae303 None None] MFA Rules not processed for user `ec2ca63a9f4c4c18bde25b1e4eab93ab`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:53:18.682378 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-50422679-489a-4e08-be84-cc42412ae303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:53:18.738021 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8356dc16-c382-4acf-9a40-48e37633afa9 service swift] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:53:18.739891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-8356dc16-c382-4acf-9a40-48e37633afa9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:53:18.765576 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8356dc16-c382-4acf-9a40-48e37633afa9 service swift] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:53:18.771084 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 971/1958] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:53:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:53:18.777756 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfca42b-9dac-44f3-9202-9d4f1c756a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:53:18.779087 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfca42b-9dac-44f3-9202-9d4f1c756a7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:18.779479 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfca42b-9dac-44f3-9202-9d4f1c756a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:18.780082 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfca42b-9dac-44f3-9202-9d4f1c756a7b admin admin] PATH_INFO: `/v3/projects/3b2d58c8957247b9ae3bfb8439a43983` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:18.781606 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfca42b-9dac-44f3-9202-9d4f1c756a7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b2d58c8957247b9ae3bfb8439a43983)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:53:18.784421 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfca42b-9dac-44f3-9202-9d4f1c756a7b admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:53:18.959038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3cfca42b-9dac-44f3-9202-9d4f1c756a7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b2d58c8957247b9ae3bfb8439a43983', 'request_id': 'req-3cfca42b-9dac-44f3-9202-9d4f1c756a7b'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:53:19.056075 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3cfca42b-9dac-44f3-9202-9d4f1c756a7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b2d58c8957247b9ae3bfb8439a43983', 'request_id': 'req-3cfca42b-9dac-44f3-9202-9d4f1c756a7b'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:53:19.072471 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3cfca42b-9dac-44f3-9202-9d4f1c756a7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b2d58c8957247b9ae3bfb8439a43983', 'request_id': 'req-3cfca42b-9dac-44f3-9202-9d4f1c756a7b'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:53:19.088088 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3cfca42b-9dac-44f3-9202-9d4f1c756a7b admin admin] The token cache is being invalidate because project 3b2d58c8957247b9ae3bfb8439a43983 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:53:19.088385 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3cfca42b-9dac-44f3-9202-9d4f1c756a7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:53:19.092241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3cfca42b-9dac-44f3-9202-9d4f1c756a7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:53:19.097854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 988/1959] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:53:18 2022] DELETE /identity/v3/projects/3b2d58c8957247b9ae3bfb8439a43983 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:53:28.761692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=94 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:53:29.084536 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=66,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:53:44.230986 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61740f-d0fb-4d32-927d-9ea2d129eddc None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:44.231516 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61740f-d0fb-4d32-927d-9ea2d129eddc None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:44.231516 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61740f-d0fb-4d32-927d-9ea2d129eddc None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:44.233120 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 972/1960] 10.222.0.17 () {54 vars in 921 bytes} [Mon Oct 17 03:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:53:44.239358 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f1ebc-3064-4953-bebd-4b0e649f9194 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:44.240581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f1ebc-3064-4953-bebd-4b0e649f9194 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:44.240581 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13f1ebc-3064-4953-bebd-4b0e649f9194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:44.261802 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-d13f1ebc-3064-4953-bebd-4b0e649f9194 None None] MFA Rules not processed for user `d961b751763a4e49b5d4ca4c795c5bf6`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:53:44.263115 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-d13f1ebc-3064-4953-bebd-4b0e649f9194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:53:44.332857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 989/1961] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Oct 17 03:53:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:53:44.344175 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61740f-d0fb-4d32-927d-9ea2d129eddc None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:53:44.345819 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-5b61740f-d0fb-4d32-927d-9ea2d129eddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:53:44.403156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61740f-d0fb-4d32-927d-9ea2d129eddc None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:53:44.408708 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb631b3-9901-4996-b18e-4fbed24d8b07 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '565a836c7f89478cad615bd6c14531a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:53:44.409392 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb631b3-9901-4996-b18e-4fbed24d8b07 service glance] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:44.409683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb631b3-9901-4996-b18e-4fbed24d8b07 service glance] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:44.409976 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb631b3-9901-4996-b18e-4fbed24d8b07 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:44.421868 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb631b3-9901-4996-b18e-4fbed24d8b07 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:53:44.423413 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb631b3-9901-4996-b18e-4fbed24d8b07 service glance] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:53:44.434864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 973/1962] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:53:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:53:44.679918 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88489f3-79d4-4913-80db-9c0a740f17f1 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:44.679918 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88489f3-79d4-4913-80db-9c0a740f17f1 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:44.679918 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88489f3-79d4-4913-80db-9c0a740f17f1 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:44.681832 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 990/1963] 10.222.0.17 () {54 vars in 921 bytes} [Mon Oct 17 03:53:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:53:44.689174 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394d0b1-fa84-46d8-9e73-8b689a4d8666 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:44.689507 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394d0b1-fa84-46d8-9e73-8b689a4d8666 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:44.689790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394d0b1-fa84-46d8-9e73-8b689a4d8666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:44.713154 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-7394d0b1-fa84-46d8-9e73-8b689a4d8666 None None] MFA Rules not processed for user `414707c545f641d3935720356b326672`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:53:44.714555 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-7394d0b1-fa84-46d8-9e73-8b689a4d8666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:53:44.769335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 974/1964] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Oct 17 03:53:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:53:44.781009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88489f3-79d4-4913-80db-9c0a740f17f1 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:53:44.782817 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-b88489f3-79d4-4913-80db-9c0a740f17f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:53:44.796881 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88489f3-79d4-4913-80db-9c0a740f17f1 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:53:44.800549 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b4ed2-5b00-4cff-9a9c-73dcba8210bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:53:44.801163 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b4ed2-5b00-4cff-9a9c-73dcba8210bc service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:44.801443 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b4ed2-5b00-4cff-9a9c-73dcba8210bc service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:44.801723 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b4ed2-5b00-4cff-9a9c-73dcba8210bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:44.809632 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6b4ed2-5b00-4cff-9a9c-73dcba8210bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:53:44.811046 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6b4ed2-5b00-4cff-9a9c-73dcba8210bc service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:53:44.819718 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 991/1965] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:53:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:53:44.915479 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a66a0-f3cc-4767-bc2d-691e9b99683c None None] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:44.916062 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a66a0-f3cc-4767-bc2d-691e9b99683c None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:44.916347 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a66a0-f3cc-4767-bc2d-691e9b99683c None None] PATH_INFO: `/` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:44.917735 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 975/1966] 10.222.0.17 () {54 vars in 921 bytes} [Mon Oct 17 03:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:53:44.923938 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a478b0-30bf-4379-a4fa-b3165f5cde70 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:44.926426 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a478b0-30bf-4379-a4fa-b3165f5cde70 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:44.926426 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a478b0-30bf-4379-a4fa-b3165f5cde70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:44.984778 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-94a478b0-30bf-4379-a4fa-b3165f5cde70 None None] MFA Rules not processed for user `d961b751763a4e49b5d4ca4c795c5bf6`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:53:44.985993 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-94a478b0-30bf-4379-a4fa-b3165f5cde70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:53:44.995272 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 992/1967] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Oct 17 03:53:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:53:45.007120 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a66a0-f3cc-4767-bc2d-691e9b99683c None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:53:45.008888 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-b11a66a0-f3cc-4767-bc2d-691e9b99683c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:53:45.022274 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a66a0-f3cc-4767-bc2d-691e9b99683c None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:53:45.025881 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7791e8-38bc-4008-8a6a-4c6635a4ba67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:53:45.026487 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7791e8-38bc-4008-8a6a-4c6635a4ba67 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:45.026814 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7791e8-38bc-4008-8a6a-4c6635a4ba67 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:45.027090 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7791e8-38bc-4008-8a6a-4c6635a4ba67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:45.037866 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7791e8-38bc-4008-8a6a-4c6635a4ba67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:53:45.039336 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7791e8-38bc-4008-8a6a-4c6635a4ba67 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:53:45.051693 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 976/1968] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:53:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:53:51.061275 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81f89c-08e7-4e15-95b9-f18e09c0aa47 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:51.061806 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81f89c-08e7-4e15-95b9-f18e09c0aa47 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:51.061905 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81f89c-08e7-4e15-95b9-f18e09c0aa47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:51.082422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-bc81f89c-08e7-4e15-95b9-f18e09c0aa47 None None] MFA Rules not processed for user `cecbc19355ef4570b2b05bba876a3a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:53:51.084186 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-bc81f89c-08e7-4e15-95b9-f18e09c0aa47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:53:51.149502 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 993/1969] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Oct 17 03:53:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:53:51.164127 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7791e8-38bc-4008-8a6a-4c6635a4ba67 service neutron] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:53:51.165613 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-0d7791e8-38bc-4008-8a6a-4c6635a4ba67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:53:51.247323 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7791e8-38bc-4008-8a6a-4c6635a4ba67 service neutron] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:53:51.252717 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100cbe7-67ee-4f1c-ba12-4b5258748914 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f64aaf4d408462a88963973d42abfb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:53:51.253393 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100cbe7-67ee-4f1c-ba12-4b5258748914 service swift] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:51.253740 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100cbe7-67ee-4f1c-ba12-4b5258748914 service swift] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:51.254075 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100cbe7-67ee-4f1c-ba12-4b5258748914 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:51.262718 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4100cbe7-67ee-4f1c-ba12-4b5258748914 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:53:51.264410 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4100cbe7-67ee-4f1c-ba12-4b5258748914 service swift] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:53:51.273466 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 977/1970] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Oct 17 03:53:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 03:53:51.576382 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81f89c-08e7-4e15-95b9-f18e09c0aa47 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:53:51.578674 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-bc81f89c-08e7-4e15-95b9-f18e09c0aa47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:53:51.667020 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81f89c-08e7-4e15-95b9-f18e09c0aa47 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:53:51.673252 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b771320-efa5-4027-8580-8b236871ea53 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b500a113a2aa4a95875d7e094b917db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:53:51.673949 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b771320-efa5-4027-8580-8b236871ea53 service placement] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:51.674229 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b771320-efa5-4027-8580-8b236871ea53 service placement] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:51.674502 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b771320-efa5-4027-8580-8b236871ea53 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:51.676440 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-8b771320-efa5-4027-8580-8b236871ea53 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:53:51.691351 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b771320-efa5-4027-8580-8b236871ea53 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:53:51.693411 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b771320-efa5-4027-8580-8b236871ea53 service placement] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:53:51.714991 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 994/1971] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:53:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:53:57.742590 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc70892-78b1-4e66-8f49-a85ee792be58 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:57.743708 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc70892-78b1-4e66-8f49-a85ee792be58 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:57.744309 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc70892-78b1-4e66-8f49-a85ee792be58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:57.766366 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-ddc70892-78b1-4e66-8f49-a85ee792be58 None None] MFA Rules not processed for user `cecbc19355ef4570b2b05bba876a3a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:53:57.770927 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-ddc70892-78b1-4e66-8f49-a85ee792be58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:53:57.779250 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 978/1972] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Oct 17 03:53:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:53:57.804978 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b771320-efa5-4027-8580-8b236871ea53 service placement] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:53:57.819040 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b771320-efa5-4027-8580-8b236871ea53 service placement] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:53:57.825190 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e73cde-6730-4785-8ee1-e28f24c3ed4c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f64aaf4d408462a88963973d42abfb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:53:57.825809 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e73cde-6730-4785-8ee1-e28f24c3ed4c service swift] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:53:57.826276 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e73cde-6730-4785-8ee1-e28f24c3ed4c service swift] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:53:57.826356 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e73cde-6730-4785-8ee1-e28f24c3ed4c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:53:57.832228 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:53:57.838795 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e73cde-6730-4785-8ee1-e28f24c3ed4c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:53:57.841154 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e73cde-6730-4785-8ee1-e28f24c3ed4c service swift] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:53:57.856339 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 995/1973] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Oct 17 03:53:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 03:54:05.080257 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218a5df-5fc5-44df-9c33-7e3a49f6fda9 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:54:05.081221 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218a5df-5fc5-44df-9c33-7e3a49f6fda9 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:54:05.081669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3218a5df-5fc5-44df-9c33-7e3a49f6fda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:54:05.104991 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-3218a5df-5fc5-44df-9c33-7e3a49f6fda9 None None] MFA Rules not processed for user `0609cf7282a949b18f3c9d9a9287baeb`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:54:05.106768 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-3218a5df-5fc5-44df-9c33-7e3a49f6fda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:54:05.225595 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:54:05.256200 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 979/1974] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:54:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:54:07.920904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:54:11.216492 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf8116-bd48-4fb5-bc06-e508bf8d849f None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:54:11.217489 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf8116-bd48-4fb5-bc06-e508bf8d849f None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:54:11.218032 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf8116-bd48-4fb5-bc06-e508bf8d849f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:54:11.238488 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-2fbf8116-bd48-4fb5-bc06-e508bf8d849f None None] MFA Rules not processed for user `cecbc19355ef4570b2b05bba876a3a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:54:11.239770 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-2fbf8116-bd48-4fb5-bc06-e508bf8d849f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:54:11.247122 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 996/1975] 10.222.0.17 () {58 vars in 967 bytes} [Mon Oct 17 03:54:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:54:11.261115 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3218a5df-5fc5-44df-9c33-7e3a49f6fda9 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:54:11.273272 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3218a5df-5fc5-44df-9c33-7e3a49f6fda9 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:54:11.278203 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38160566-764a-4061-95d6-d8a29ecb688f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f64aaf4d408462a88963973d42abfb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:54:11.278916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38160566-764a-4061-95d6-d8a29ecb688f service swift] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:54:11.279197 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38160566-764a-4061-95d6-d8a29ecb688f service swift] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:54:11.279468 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38160566-764a-4061-95d6-d8a29ecb688f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:54:11.287295 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38160566-764a-4061-95d6-d8a29ecb688f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:54:11.288889 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38160566-764a-4061-95d6-d8a29ecb688f service swift] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:54:11.296825 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 980/1976] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Oct 17 03:54:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 03:54:14.585916 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2cffe-15d1-4ae4-9f78-e71f7dec4bc9 None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:54:14.586451 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2cffe-15d1-4ae4-9f78-e71f7dec4bc9 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:54:14.586559 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef2cffe-15d1-4ae4-9f78-e71f7dec4bc9 None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:54:14.588127 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 997/1977] 10.222.0.17 () {54 vars in 921 bytes} [Mon Oct 17 03:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:54:14.617777 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f8ca0-cf77-420e-977c-dcb66fb1fcbe None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:54:14.618340 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f8ca0-cf77-420e-977c-dcb66fb1fcbe None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:54:14.618340 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f8ca0-cf77-420e-977c-dcb66fb1fcbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:54:14.639140 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-711f8ca0-cf77-420e-977c-dcb66fb1fcbe None None] MFA Rules not processed for user `b500a113a2aa4a95875d7e094b917db3`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:54:14.640384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-711f8ca0-cf77-420e-977c-dcb66fb1fcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:54:14.649234 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 981/1978] 10.222.0.17 () {58 vars in 1018 bytes} [Mon Oct 17 03:54:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:54:14.663443 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70254ef4-2ebf-4344-9cd5-7069b2340f7e None None] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:54:14.663859 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70254ef4-2ebf-4344-9cd5-7069b2340f7e None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:54:14.664138 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70254ef4-2ebf-4344-9cd5-7069b2340f7e None None] PATH_INFO: `/` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:54:14.665510 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 998/1979] 10.222.0.17 () {54 vars in 961 bytes} [Mon Oct 17 03:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 17 03:54:14.670835 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17e5dd-82a7-4b46-9c49-19eacfe2337c None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:54:14.671207 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17e5dd-82a7-4b46-9c49-19eacfe2337c None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:54:14.671548 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17e5dd-82a7-4b46-9c49-19eacfe2337c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:54:14.698702 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-4d17e5dd-82a7-4b46-9c49-19eacfe2337c None None] MFA Rules not processed for user `b500a113a2aa4a95875d7e094b917db3`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:54:14.699922 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-4d17e5dd-82a7-4b46-9c49-19eacfe2337c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:54:14.709332 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 982/1980] 10.222.0.17 () {58 vars in 1058 bytes} [Mon Oct 17 03:54:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:54:14.716842 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70254ef4-2ebf-4344-9cd5-7069b2340f7e None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:54:14.730401 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70254ef4-2ebf-4344-9cd5-7069b2340f7e None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:54:14.740792 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de750110-e119-4a7c-b584-d3d508fdc6f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b500a113a2aa4a95875d7e094b917db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:54:14.741436 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de750110-e119-4a7c-b584-d3d508fdc6f6 service placement] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:54:14.741716 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de750110-e119-4a7c-b584-d3d508fdc6f6 service placement] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:54:14.742396 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de750110-e119-4a7c-b584-d3d508fdc6f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:54:14.751863 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de750110-e119-4a7c-b584-d3d508fdc6f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:54:14.756609 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de750110-e119-4a7c-b584-d3d508fdc6f6 service placement] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:54:14.780932 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 999/1981] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:54:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:54:14.781153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17e5dd-82a7-4b46-9c49-19eacfe2337c None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:54:14.781826 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-4d17e5dd-82a7-4b46-9c49-19eacfe2337c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:54:14.868040 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17e5dd-82a7-4b46-9c49-19eacfe2337c None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:54:14.872878 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc31490-e614-4c13-a2cb-fc995c7d797d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:54:14.873648 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc31490-e614-4c13-a2cb-fc995c7d797d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:54:14.874029 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc31490-e614-4c13-a2cb-fc995c7d797d admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:54:14.874405 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc31490-e614-4c13-a2cb-fc995c7d797d admin admin] PATH_INFO: `/v3/users/1200fc635ffa41caa5e46cacaa66b5aa` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:54:14.875788 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc31490-e614-4c13-a2cb-fc995c7d797d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1200fc635ffa41caa5e46cacaa66b5aa)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:54:14.877911 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc31490-e614-4c13-a2cb-fc995c7d797d admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:54:15.072086 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-8cc31490-e614-4c13-a2cb-fc995c7d797d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1200fc635ffa41caa5e46cacaa66b5aa'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:54:15.144725 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-8cc31490-e614-4c13-a2cb-fc995c7d797d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1200fc635ffa41caa5e46cacaa66b5aa'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:54:15.157882 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-8cc31490-e614-4c13-a2cb-fc995c7d797d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1200fc635ffa41caa5e46cacaa66b5aa'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:54:15.158575 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-8cc31490-e614-4c13-a2cb-fc995c7d797d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1200fc635ffa41caa5e46cacaa66b5aa'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:54:15.159205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-8cc31490-e614-4c13-a2cb-fc995c7d797d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1200fc635ffa41caa5e46cacaa66b5aa'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:54:15.172511 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 983/1982] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:54:14 2022] DELETE /identity/v3/users/1200fc635ffa41caa5e46cacaa66b5aa => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:54:15.435186 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de750110-e119-4a7c-b584-d3d508fdc6f6 service placement] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:54:15.453527 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de750110-e119-4a7c-b584-d3d508fdc6f6 service placement] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:54:15.512596 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef65f39-ddd1-4596-9f0c-c66e6913c274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:54:15.513219 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef65f39-ddd1-4596-9f0c-c66e6913c274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:54:15.513497 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef65f39-ddd1-4596-9f0c-c66e6913c274 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:54:15.513767 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef65f39-ddd1-4596-9f0c-c66e6913c274 admin admin] PATH_INFO: `/v3/projects/25d7abaaf76a4147afb1acb1ed2385b2` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:54:15.514870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef65f39-ddd1-4596-9f0c-c66e6913c274 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25d7abaaf76a4147afb1acb1ed2385b2)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:54:15.516784 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef65f39-ddd1-4596-9f0c-c66e6913c274 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:54:15.617338 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-1ef65f39-ddd1-4596-9f0c-c66e6913c274 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25d7abaaf76a4147afb1acb1ed2385b2', 'request_id': 'req-1ef65f39-ddd1-4596-9f0c-c66e6913c274'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:54:15.678818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-1ef65f39-ddd1-4596-9f0c-c66e6913c274 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25d7abaaf76a4147afb1acb1ed2385b2', 'request_id': 'req-1ef65f39-ddd1-4596-9f0c-c66e6913c274'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:54:15.684762 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-1ef65f39-ddd1-4596-9f0c-c66e6913c274 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25d7abaaf76a4147afb1acb1ed2385b2', 'request_id': 'req-1ef65f39-ddd1-4596-9f0c-c66e6913c274'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:54:15.696352 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-1ef65f39-ddd1-4596-9f0c-c66e6913c274 admin admin] The token cache is being invalidate because project 25d7abaaf76a4147afb1acb1ed2385b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:54:15.696641 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-1ef65f39-ddd1-4596-9f0c-c66e6913c274 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:54:15.697276 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-1ef65f39-ddd1-4596-9f0c-c66e6913c274 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:54:15.698800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1000/1983] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:54:15 2022] DELETE /identity/v3/projects/25d7abaaf76a4147afb1acb1ed2385b2 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:54:21.033322 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff544f-f873-423f-ab65-255ac2cbeb87 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:54:21.033807 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff544f-f873-423f-ab65-255ac2cbeb87 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:54:21.033920 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcff544f-f873-423f-ab65-255ac2cbeb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:54:21.057588 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-bcff544f-f873-423f-ab65-255ac2cbeb87 None None] MFA Rules not processed for user `d1499b45cce347c39855da14d66e9690`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:54:21.059533 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-bcff544f-f873-423f-ab65-255ac2cbeb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:54:21.083063 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:54:21.129466 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 984/1984] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:54:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:54:21.576753 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1aa57-0347-4e9b-b06b-620fb1666c38 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:54:21.577239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1aa57-0347-4e9b-b06b-620fb1666c38 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:54:21.577500 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1aa57-0347-4e9b-b06b-620fb1666c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:54:21.598224 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-5bd1aa57-0347-4e9b-b06b-620fb1666c38 None None] MFA Rules not processed for user `cecbc19355ef4570b2b05bba876a3a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:54:21.600777 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-5bd1aa57-0347-4e9b-b06b-620fb1666c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:54:21.660330 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1001/1985] 10.222.0.17 () {58 vars in 967 bytes} [Mon Oct 17 03:54:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:54:21.690868 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff544f-f873-423f-ab65-255ac2cbeb87 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:54:21.691629 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-bcff544f-f873-423f-ab65-255ac2cbeb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:54:21.759619 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcff544f-f873-423f-ab65-255ac2cbeb87 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:54:21.764517 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02887df3-f5ad-4cf6-aa43-84bc483363a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f64aaf4d408462a88963973d42abfb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:54:21.765136 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02887df3-f5ad-4cf6-aa43-84bc483363a9 service swift] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:54:21.765394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02887df3-f5ad-4cf6-aa43-84bc483363a9 service swift] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:54:21.765667 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02887df3-f5ad-4cf6-aa43-84bc483363a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:54:21.774223 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02887df3-f5ad-4cf6-aa43-84bc483363a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:54:21.775812 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02887df3-f5ad-4cf6-aa43-84bc483363a9 service swift] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:54:21.783770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 985/1986] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Oct 17 03:54:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 03:54:31.653465 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:54:31.780491 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=48 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:55:11.302997 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ddee2-7aa3-4c36-8fa6-ba7a663aeddf None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:55:11.303549 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ddee2-7aa3-4c36-8fa6-ba7a663aeddf None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:55:11.303549 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ddee2-7aa3-4c36-8fa6-ba7a663aeddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:55:11.323099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-b05ddee2-7aa3-4c36-8fa6-ba7a663aeddf None None] MFA Rules not processed for user `ec2ca63a9f4c4c18bde25b1e4eab93ab`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:55:11.324889 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-b05ddee2-7aa3-4c36-8fa6-ba7a663aeddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:55:11.379965 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1002/1987] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:55:11 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:55:20.407634 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02887df3-f5ad-4cf6-aa43-84bc483363a9 service swift] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:55:20.410267 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-02887df3-f5ad-4cf6-aa43-84bc483363a9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:55:20.483600 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02887df3-f5ad-4cf6-aa43-84bc483363a9 service swift] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:55:20.489350 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37ba42-157c-4aa1-ac84-82435c36c64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:55:20.490408 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37ba42-157c-4aa1-ac84-82435c36c64c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:55:20.490905 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37ba42-157c-4aa1-ac84-82435c36c64c admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:55:20.491493 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37ba42-157c-4aa1-ac84-82435c36c64c admin admin] PATH_INFO: `/v3/users/0609cf7282a949b18f3c9d9a9287baeb` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:55:20.492995 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37ba42-157c-4aa1-ac84-82435c36c64c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0609cf7282a949b18f3c9d9a9287baeb)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:55:20.496750 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37ba42-157c-4aa1-ac84-82435c36c64c admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:55:20.684681 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-4c37ba42-157c-4aa1-ac84-82435c36c64c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0609cf7282a949b18f3c9d9a9287baeb'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:55:20.773255 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-4c37ba42-157c-4aa1-ac84-82435c36c64c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0609cf7282a949b18f3c9d9a9287baeb'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:55:20.787691 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-4c37ba42-157c-4aa1-ac84-82435c36c64c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0609cf7282a949b18f3c9d9a9287baeb'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:55:20.788526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-4c37ba42-157c-4aa1-ac84-82435c36c64c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0609cf7282a949b18f3c9d9a9287baeb'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:55:20.789204 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-4c37ba42-157c-4aa1-ac84-82435c36c64c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0609cf7282a949b18f3c9d9a9287baeb'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:55:20.804428 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 986/1988] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:55:20 2022] DELETE /identity/v3/users/0609cf7282a949b18f3c9d9a9287baeb => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:55:21.092105 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ddee2-7aa3-4c36-8fa6-ba7a663aeddf None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:55:21.109782 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05ddee2-7aa3-4c36-8fa6-ba7a663aeddf None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:55:21.199772 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7473feb-4db6-44b1-8a62-491033514327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:55:21.200456 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7473feb-4db6-44b1-8a62-491033514327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:55:21.200746 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7473feb-4db6-44b1-8a62-491033514327 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:55:21.201024 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7473feb-4db6-44b1-8a62-491033514327 admin admin] PATH_INFO: `/v3/projects/c561fe5da6e644839d762e7ecef5515c` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:55:21.202209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7473feb-4db6-44b1-8a62-491033514327 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c561fe5da6e644839d762e7ecef5515c)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:55:21.204936 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7473feb-4db6-44b1-8a62-491033514327 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:55:21.334854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d7473feb-4db6-44b1-8a62-491033514327 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c561fe5da6e644839d762e7ecef5515c', 'request_id': 'req-d7473feb-4db6-44b1-8a62-491033514327'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:55:21.409168 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d7473feb-4db6-44b1-8a62-491033514327 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c561fe5da6e644839d762e7ecef5515c', 'request_id': 'req-d7473feb-4db6-44b1-8a62-491033514327'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:55:21.415222 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d7473feb-4db6-44b1-8a62-491033514327 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c561fe5da6e644839d762e7ecef5515c', 'request_id': 'req-d7473feb-4db6-44b1-8a62-491033514327'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:55:21.433311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d7473feb-4db6-44b1-8a62-491033514327 admin admin] The token cache is being invalidate because project c561fe5da6e644839d762e7ecef5515c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:55:21.433311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d7473feb-4db6-44b1-8a62-491033514327 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:55:21.433311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d7473feb-4db6-44b1-8a62-491033514327 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:55:21.433311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1003/1989] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:55:21 2022] DELETE /identity/v3/projects/c561fe5da6e644839d762e7ecef5515c => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:55:25.097698 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8e71f-9451-4116-b24b-27862e852bb2 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:55:25.098189 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8e71f-9451-4116-b24b-27862e852bb2 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:55:25.098189 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8e71f-9451-4116-b24b-27862e852bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:55:25.117937 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-34f8e71f-9451-4116-b24b-27862e852bb2 None None] MFA Rules not processed for user `57ee3a13e33549c9a90ebde353fcd6f2`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:55:25.119123 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-34f8e71f-9451-4116-b24b-27862e852bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:55:25.173269 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 987/1990] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:55:25 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:55:31.423486 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=76,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:55:35.166690 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=78,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:55:42.749061 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe5259-b0c9-4316-8a19-24351c1cdd1d None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:55:42.749567 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe5259-b0c9-4316-8a19-24351c1cdd1d None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:55:42.749672 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fe5259-b0c9-4316-8a19-24351c1cdd1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:55:42.769867 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-40fe5259-b0c9-4316-8a19-24351c1cdd1d None None] MFA Rules not processed for user `d1499b45cce347c39855da14d66e9690`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:55:42.771053 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-40fe5259-b0c9-4316-8a19-24351c1cdd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:55:42.824640 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1004/1991] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:55:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:55:42.862472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8e71f-9451-4116-b24b-27862e852bb2 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:55:42.863922 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-34f8e71f-9451-4116-b24b-27862e852bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:55:42.918552 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f8e71f-9451-4116-b24b-27862e852bb2 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:55:42.922040 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c69f0d-b0a6-4bce-b8d8-7956f49b8ddd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '565a836c7f89478cad615bd6c14531a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:55:42.922633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c69f0d-b0a6-4bce-b8d8-7956f49b8ddd service glance] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:55:42.922908 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c69f0d-b0a6-4bce-b8d8-7956f49b8ddd service glance] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:55:42.923182 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c69f0d-b0a6-4bce-b8d8-7956f49b8ddd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:55:42.925619 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c8c69f0d-b0a6-4bce-b8d8-7956f49b8ddd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:55:42.993779 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c69f0d-b0a6-4bce-b8d8-7956f49b8ddd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:55:42.995191 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c69f0d-b0a6-4bce-b8d8-7956f49b8ddd service glance] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:55:43.005747 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 988/1992] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:55:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:55:43.678780 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe5259-b0c9-4316-8a19-24351c1cdd1d None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:55:43.680761 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-40fe5259-b0c9-4316-8a19-24351c1cdd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:55:43.745919 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fe5259-b0c9-4316-8a19-24351c1cdd1d None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:55:43.749706 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6b881-0f80-4db2-8c68-4135a52618d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:55:43.750329 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6b881-0f80-4db2-8c68-4135a52618d2 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:55:43.750608 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6b881-0f80-4db2-8c68-4135a52618d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:55:43.751080 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6b881-0f80-4db2-8c68-4135a52618d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:55:43.753422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-a7a6b881-0f80-4db2-8c68-4135a52618d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:55:43.767374 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6b881-0f80-4db2-8c68-4135a52618d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:55:43.769408 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6b881-0f80-4db2-8c68-4135a52618d2 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:55:43.781611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1005/1993] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:55:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:55:52.565051 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c69f0d-b0a6-4bce-b8d8-7956f49b8ddd service glance] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:55:52.566422 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c8c69f0d-b0a6-4bce-b8d8-7956f49b8ddd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:55:52.622048 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c69f0d-b0a6-4bce-b8d8-7956f49b8ddd service glance] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:55:52.625536 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517ef1a-669a-4fd5-b197-df238f0a8b3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:55:52.626132 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517ef1a-669a-4fd5-b197-df238f0a8b3a service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:55:52.626407 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517ef1a-669a-4fd5-b197-df238f0a8b3a service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:55:52.626708 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517ef1a-669a-4fd5-b197-df238f0a8b3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:55:52.628364 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-f517ef1a-669a-4fd5-b197-df238f0a8b3a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:55:52.682722 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:55:52.685235 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f517ef1a-669a-4fd5-b197-df238f0a8b3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:55:52.686636 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f517ef1a-669a-4fd5-b197-df238f0a8b3a service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:55:52.695586 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 989/1994] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:55:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:55:53.776413 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=51 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:55:59.948839 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6b881-0f80-4db2-8c68-4135a52618d2 service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:55:59.950296 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-a7a6b881-0f80-4db2-8c68-4135a52618d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:55:59.969576 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6b881-0f80-4db2-8c68-4135a52618d2 service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:55:59.979149 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0862638-5341-4c38-9a24-7ad28e1c43a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:55:59.980124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0862638-5341-4c38-9a24-7ad28e1c43a7 service nova] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:55:59.980611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0862638-5341-4c38-9a24-7ad28e1c43a7 service nova] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:55:59.981101 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0862638-5341-4c38-9a24-7ad28e1c43a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:55:59.983105 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-c0862638-5341-4c38-9a24-7ad28e1c43a7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:00.063210 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0862638-5341-4c38-9a24-7ad28e1c43a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:00.065602 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0862638-5341-4c38-9a24-7ad28e1c43a7 service nova] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:00.076480 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1006/1995] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:55:59 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:56:02.715441 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=2 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:05.834842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8acfb-6eba-4910-9b61-e2266eb70a5b None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:05.835445 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8acfb-6eba-4910-9b61-e2266eb70a5b None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:05.835445 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd8acfb-6eba-4910-9b61-e2266eb70a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:05.854140 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-acd8acfb-6eba-4910-9b61-e2266eb70a5b None None] MFA Rules not processed for user `6f1245ffc45b4c99990e746068bd852c`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:56:05.855329 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-acd8acfb-6eba-4910-9b61-e2266eb70a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:05.863319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 990/1996] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:56:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:56:06.170927 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0862638-5341-4c38-9a24-7ad28e1c43a7 service nova] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:06.180684 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0862638-5341-4c38-9a24-7ad28e1c43a7 service nova] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:06.184424 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb70db-4ef8-4fe2-b6c4-96a59fc53dbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:06.188072 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb70db-4ef8-4fe2-b6c4-96a59fc53dbd service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:06.188072 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb70db-4ef8-4fe2-b6c4-96a59fc53dbd service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:06.188072 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb70db-4ef8-4fe2-b6c4-96a59fc53dbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:06.188072 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-f7eb70db-4ef8-4fe2-b6c4-96a59fc53dbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:06.196563 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb70db-4ef8-4fe2-b6c4-96a59fc53dbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:06.197962 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb70db-4ef8-4fe2-b6c4-96a59fc53dbd service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:06.205775 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1007/1997] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:56:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:56:06.637511 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8acfb-6eba-4910-9b61-e2266eb70a5b None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:06.639117 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-acd8acfb-6eba-4910-9b61-e2266eb70a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:06.711297 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd8acfb-6eba-4910-9b61-e2266eb70a5b None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:06.716205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690e8cd-e080-482b-ab2e-eb4cc193adfc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b500a113a2aa4a95875d7e094b917db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:06.716817 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690e8cd-e080-482b-ab2e-eb4cc193adfc service placement] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:06.717098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690e8cd-e080-482b-ab2e-eb4cc193adfc service placement] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:06.717371 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8690e8cd-e080-482b-ab2e-eb4cc193adfc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:06.719069 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-8690e8cd-e080-482b-ab2e-eb4cc193adfc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:06.731182 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690e8cd-e080-482b-ab2e-eb4cc193adfc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:06.733001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8690e8cd-e080-482b-ab2e-eb4cc193adfc service placement] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:06.743014 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 991/1998] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:56:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:56:09.732059 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb70db-4ef8-4fe2-b6c4-96a59fc53dbd service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:09.740962 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb70db-4ef8-4fe2-b6c4-96a59fc53dbd service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:09.744752 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c4541-d728-4e34-adb4-3c9497173eac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:09.745368 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c4541-d728-4e34-adb4-3c9497173eac service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:09.745652 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c4541-d728-4e34-adb4-3c9497173eac service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:09.745902 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772c4541-d728-4e34-adb4-3c9497173eac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:09.747703 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-772c4541-d728-4e34-adb4-3c9497173eac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:09.757668 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c4541-d728-4e34-adb4-3c9497173eac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:09.759082 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772c4541-d728-4e34-adb4-3c9497173eac service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:09.767085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1008/1999] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:56:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:56:14.430726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690e8cd-e080-482b-ab2e-eb4cc193adfc service placement] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:14.432176 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-8690e8cd-e080-482b-ab2e-eb4cc193adfc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:14.447202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8690e8cd-e080-482b-ab2e-eb4cc193adfc service placement] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:14.452708 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0117b4-d4a5-4e1d-a1ab-d5b60b1c86bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:14.453299 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0117b4-d4a5-4e1d-a1ab-d5b60b1c86bf service nova] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:14.453576 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0117b4-d4a5-4e1d-a1ab-d5b60b1c86bf service nova] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:14.453845 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0117b4-d4a5-4e1d-a1ab-d5b60b1c86bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:14.456071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-6e0117b4-d4a5-4e1d-a1ab-d5b60b1c86bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:14.467841 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0117b4-d4a5-4e1d-a1ab-d5b60b1c86bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:14.469388 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0117b4-d4a5-4e1d-a1ab-d5b60b1c86bf service nova] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:14.480001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 992/2000] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:56:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:56:17.317661 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c4541-d728-4e34-adb4-3c9497173eac service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:17.319047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-772c4541-d728-4e34-adb4-3c9497173eac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:17.329131 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772c4541-d728-4e34-adb4-3c9497173eac service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:17.333643 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f3380-43c3-42de-bb09-e99d78512001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:17.334230 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f3380-43c3-42de-bb09-e99d78512001 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:17.334507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f3380-43c3-42de-bb09-e99d78512001 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:17.334776 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f3380-43c3-42de-bb09-e99d78512001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:17.336404 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-fc7f3380-43c3-42de-bb09-e99d78512001 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:17.398914 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f3380-43c3-42de-bb09-e99d78512001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:17.400412 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f3380-43c3-42de-bb09-e99d78512001 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:17.410295 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1009/2001] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:56:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:56:21.408038 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0117b4-d4a5-4e1d-a1ab-d5b60b1c86bf service nova] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:21.409957 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-6e0117b4-d4a5-4e1d-a1ab-d5b60b1c86bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:21.423923 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0117b4-d4a5-4e1d-a1ab-d5b60b1c86bf service nova] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:21.429507 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd8267-a59a-46ab-a372-f3ef92f04a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:21.430131 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd8267-a59a-46ab-a372-f3ef92f04a36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:21.430414 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd8267-a59a-46ab-a372-f3ef92f04a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:21.430687 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd8267-a59a-46ab-a372-f3ef92f04a36 admin admin] PATH_INFO: `/v3/users/ec2ca63a9f4c4c18bde25b1e4eab93ab` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:21.431844 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dd8267-a59a-46ab-a372-f3ef92f04a36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec2ca63a9f4c4c18bde25b1e4eab93ab)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:21.434968 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dd8267-a59a-46ab-a372-f3ef92f04a36 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:21.611680 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c9dd8267-a59a-46ab-a372-f3ef92f04a36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec2ca63a9f4c4c18bde25b1e4eab93ab'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:21.689885 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c9dd8267-a59a-46ab-a372-f3ef92f04a36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec2ca63a9f4c4c18bde25b1e4eab93ab'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:21.702679 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c9dd8267-a59a-46ab-a372-f3ef92f04a36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec2ca63a9f4c4c18bde25b1e4eab93ab'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:21.703410 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c9dd8267-a59a-46ab-a372-f3ef92f04a36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec2ca63a9f4c4c18bde25b1e4eab93ab'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:21.704037 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c9dd8267-a59a-46ab-a372-f3ef92f04a36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec2ca63a9f4c4c18bde25b1e4eab93ab'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:21.717061 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 993/2002] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:56:21 2022] DELETE /identity/v3/users/ec2ca63a9f4c4c18bde25b1e4eab93ab => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:56:21.948827 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f3380-43c3-42de-bb09-e99d78512001 service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:21.960654 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f3380-43c3-42de-bb09-e99d78512001 service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:22.018111 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c2ac3-897d-41b7-b81c-a76ce1b3f895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:22.019253 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c2ac3-897d-41b7-b81c-a76ce1b3f895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:22.019539 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c2ac3-897d-41b7-b81c-a76ce1b3f895 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:22.019847 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c2ac3-897d-41b7-b81c-a76ce1b3f895 admin admin] PATH_INFO: `/v3/projects/cf238531f665437faddc8002d3b2ab6c` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:22.021035 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c2ac3-897d-41b7-b81c-a76ce1b3f895 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf238531f665437faddc8002d3b2ab6c)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:22.022955 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c2ac3-897d-41b7-b81c-a76ce1b3f895 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:22.031837 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:22.137202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d95c2ac3-897d-41b7-b81c-a76ce1b3f895 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf238531f665437faddc8002d3b2ab6c', 'request_id': 'req-d95c2ac3-897d-41b7-b81c-a76ce1b3f895'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:22.199032 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d95c2ac3-897d-41b7-b81c-a76ce1b3f895 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf238531f665437faddc8002d3b2ab6c', 'request_id': 'req-d95c2ac3-897d-41b7-b81c-a76ce1b3f895'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:22.205487 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d95c2ac3-897d-41b7-b81c-a76ce1b3f895 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf238531f665437faddc8002d3b2ab6c', 'request_id': 'req-d95c2ac3-897d-41b7-b81c-a76ce1b3f895'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:22.217239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d95c2ac3-897d-41b7-b81c-a76ce1b3f895 admin admin] The token cache is being invalidate because project cf238531f665437faddc8002d3b2ab6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:56:22.217762 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d95c2ac3-897d-41b7-b81c-a76ce1b3f895 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:22.218578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-d95c2ac3-897d-41b7-b81c-a76ce1b3f895 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:22.220339 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1010/2003] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:56:21 2022] DELETE /identity/v3/projects/cf238531f665437faddc8002d3b2ab6c => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:56:22.225321 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58cdbec-3238-4f66-bf6a-fd1019ac5761 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:22.225686 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58cdbec-3238-4f66-bf6a-fd1019ac5761 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:22.227461 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58cdbec-3238-4f66-bf6a-fd1019ac5761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:22.246425 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-e58cdbec-3238-4f66-bf6a-fd1019ac5761 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:56:22.247643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-e58cdbec-3238-4f66-bf6a-fd1019ac5761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:22.288954 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:22.309963 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 994/2004] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:56:22.318276 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61e021-1b26-4c84-a5a3-cabc26349de1 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:22.318646 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61e021-1b26-4c84-a5a3-cabc26349de1 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:22.319013 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc61e021-1b26-4c84-a5a3-cabc26349de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:22.339423 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-cc61e021-1b26-4c84-a5a3-cabc26349de1 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:56:22.340792 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-cc61e021-1b26-4c84-a5a3-cabc26349de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:22.351989 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1011/2005] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:56:22.353299 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58cdbec-3238-4f66-bf6a-fd1019ac5761 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:22.363437 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58cdbec-3238-4f66-bf6a-fd1019ac5761 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:22.368243 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5495da-0bf2-4da4-8432-7f2c5b6cac3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:22.368851 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5495da-0bf2-4da4-8432-7f2c5b6cac3f admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:22.369138 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5495da-0bf2-4da4-8432-7f2c5b6cac3f admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:22.369449 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5495da-0bf2-4da4-8432-7f2c5b6cac3f admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:22.370097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5495da-0bf2-4da4-8432-7f2c5b6cac3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:56:22.370484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5495da-0bf2-4da4-8432-7f2c5b6cac3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:22.372119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5495da-0bf2-4da4-8432-7f2c5b6cac3f admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:22.384764 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 995/2006] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:56:22.389270 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61e021-1b26-4c84-a5a3-cabc26349de1 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:22.398589 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc61e021-1b26-4c84-a5a3-cabc26349de1 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:22.403313 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66019536-831b-43ee-908e-4145cc21b484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:22.403959 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66019536-831b-43ee-908e-4145cc21b484 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:22.404385 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66019536-831b-43ee-908e-4145cc21b484 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:22.404655 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66019536-831b-43ee-908e-4145cc21b484 admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:22.405284 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66019536-831b-43ee-908e-4145cc21b484 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:22.407201 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66019536-831b-43ee-908e-4145cc21b484 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:22.510124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1012/2007] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:56:22 2022] POST /identity/v3/projects => generated 405 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:56:22.511449 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5495da-0bf2-4da4-8432-7f2c5b6cac3f admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:22.522144 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5495da-0bf2-4da4-8432-7f2c5b6cac3f admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:22.578844 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a845c38-dd48-4c8f-8ff8-626e4497700f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:22.579546 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a845c38-dd48-4c8f-8ff8-626e4497700f admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:22.579872 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a845c38-dd48-4c8f-8ff8-626e4497700f admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:22.580148 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a845c38-dd48-4c8f-8ff8-626e4497700f admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:22.580840 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a845c38-dd48-4c8f-8ff8-626e4497700f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:22.582741 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a845c38-dd48-4c8f-8ff8-626e4497700f admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:22.665428 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 996/2008] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:56:22 2022] POST /identity/v3/users => generated 303 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:56:22.666930 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66019536-831b-43ee-908e-4145cc21b484 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:22.678175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66019536-831b-43ee-908e-4145cc21b484 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:22.682909 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654153c9-60f8-4748-b497-b5adb3bc42fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:22.684934 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654153c9-60f8-4748-b497-b5adb3bc42fa admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:22.684934 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654153c9-60f8-4748-b497-b5adb3bc42fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:22.684934 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654153c9-60f8-4748-b497-b5adb3bc42fa admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:22.684934 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654153c9-60f8-4748-b497-b5adb3bc42fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:22.686401 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654153c9-60f8-4748-b497-b5adb3bc42fa admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:22.697059 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1013/2009] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:56:22.703534 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a845c38-dd48-4c8f-8ff8-626e4497700f admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:22.713044 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a845c38-dd48-4c8f-8ff8-626e4497700f admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:22.718044 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a926ea-311a-40b5-bd1b-a65b5820d0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:22.718685 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a926ea-311a-40b5-bd1b-a65b5820d0a2 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:22.718964 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a926ea-311a-40b5-bd1b-a65b5820d0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:22.719250 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a926ea-311a-40b5-bd1b-a65b5820d0a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:22.719918 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a926ea-311a-40b5-bd1b-a65b5820d0a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:22.725142 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a926ea-311a-40b5-bd1b-a65b5820d0a2 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:22.733166 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 997/2010] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:56:22.738042 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654153c9-60f8-4748-b497-b5adb3bc42fa admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:22.751010 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654153c9-60f8-4748-b497-b5adb3bc42fa admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:22.755875 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c37be-512b-4fd9-b0af-99c6f4358f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:22.756555 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c37be-512b-4fd9-b0af-99c6f4358f73 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:22.756832 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c37be-512b-4fd9-b0af-99c6f4358f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:22.757101 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c37be-512b-4fd9-b0af-99c6f4358f73 admin admin] PATH_INFO: `/v3/projects/383031080dc540529bc1e8f4e190a347/users/81ca8826c48c4bd698743babd31e0d19/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:22.772183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c37be-512b-4fd9-b0af-99c6f4358f73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=383031080dc540529bc1e8f4e190a347, user_id=81ca8826c48c4bd698743babd31e0d19, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:22.776776 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c37be-512b-4fd9-b0af-99c6f4358f73 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:22.858917 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1014/2011] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:56:22 2022] PUT /identity/v3/projects/383031080dc540529bc1e8f4e190a347/users/81ca8826c48c4bd698743babd31e0d19/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:56:22.869926 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc0a0b-3b47-4b2f-94bf-08275cf1ca28 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:22.870230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc0a0b-3b47-4b2f-94bf-08275cf1ca28 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:22.870505 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc0a0b-3b47-4b2f-94bf-08275cf1ca28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:22.898728 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-7efc0a0b-3b47-4b2f-94bf-08275cf1ca28 None None] MFA Rules not processed for user `81ca8826c48c4bd698743babd31e0d19`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:56:22.899993 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-7efc0a0b-3b47-4b2f-94bf-08275cf1ca28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:22.965870 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 998/2012] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:56:22 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:56:22.972075 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c37be-512b-4fd9-b0af-99c6f4358f73 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:23.033455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c37be-512b-4fd9-b0af-99c6f4358f73 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:23.038685 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f35e1-acb2-49e4-870f-714b2b423c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:23.039295 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f35e1-acb2-49e4-870f-714b2b423c11 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:23.039578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f35e1-acb2-49e4-870f-714b2b423c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:23.039885 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f35e1-acb2-49e4-870f-714b2b423c11 admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:23.040544 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f35e1-acb2-49e4-870f-714b2b423c11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:23.042463 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f35e1-acb2-49e4-870f-714b2b423c11 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:23.126245 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1015/2013] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:56:22 2022] POST /identity/v3/projects => generated 405 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:56:23.130570 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc0a0b-3b47-4b2f-94bf-08275cf1ca28 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:23.141263 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc0a0b-3b47-4b2f-94bf-08275cf1ca28 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:23.176432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:23.199585 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ca70f-e11c-41be-989e-001a6e07d4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:23.200236 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ca70f-e11c-41be-989e-001a6e07d4fc admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:23.203573 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ca70f-e11c-41be-989e-001a6e07d4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:23.203573 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ca70f-e11c-41be-989e-001a6e07d4fc admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:23.203573 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3ca70f-e11c-41be-989e-001a6e07d4fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:23.203573 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3ca70f-e11c-41be-989e-001a6e07d4fc admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:23.316844 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 999/2014] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:56:23 2022] POST /identity/v3/users => generated 309 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:56:23.318607 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f35e1-acb2-49e4-870f-714b2b423c11 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:23.329643 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f35e1-acb2-49e4-870f-714b2b423c11 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:23.335526 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77adde-1b8f-4b06-8849-52f7d31b232e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:23.336241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77adde-1b8f-4b06-8849-52f7d31b232e admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:23.340323 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77adde-1b8f-4b06-8849-52f7d31b232e admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:23.340323 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77adde-1b8f-4b06-8849-52f7d31b232e admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:23.340323 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77adde-1b8f-4b06-8849-52f7d31b232e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:23.340323 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77adde-1b8f-4b06-8849-52f7d31b232e admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:23.355110 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1016/2015] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:56:23.356093 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ca70f-e11c-41be-989e-001a6e07d4fc admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:23.367354 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ca70f-e11c-41be-989e-001a6e07d4fc admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:23.373024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7068e0-53e6-42ec-bf0b-57f643b44ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:23.373820 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7068e0-53e6-42ec-bf0b-57f643b44ba1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:23.374221 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7068e0-53e6-42ec-bf0b-57f643b44ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:23.374600 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7068e0-53e6-42ec-bf0b-57f643b44ba1 admin admin] PATH_INFO: `/v3/projects/165d26e917314feab1cc0f428c5cc0c1/users/b643550a47cf4fc8a750b5b4411ded1b/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:23.392684 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7068e0-53e6-42ec-bf0b-57f643b44ba1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=165d26e917314feab1cc0f428c5cc0c1, user_id=b643550a47cf4fc8a750b5b4411ded1b, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:23.396462 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7068e0-53e6-42ec-bf0b-57f643b44ba1 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:23.506657 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1000/2016] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:56:23 2022] PUT /identity/v3/projects/165d26e917314feab1cc0f428c5cc0c1/users/b643550a47cf4fc8a750b5b4411ded1b/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:56:23.507922 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77adde-1b8f-4b06-8849-52f7d31b232e admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:23.519795 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77adde-1b8f-4b06-8849-52f7d31b232e admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:23.523290 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:23.581646 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893eaed5-3c3b-4fb1-829b-a98b32570292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:23.582359 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893eaed5-3c3b-4fb1-829b-a98b32570292 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:23.582738 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893eaed5-3c3b-4fb1-829b-a98b32570292 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:23.583081 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893eaed5-3c3b-4fb1-829b-a98b32570292 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:23.584086 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893eaed5-3c3b-4fb1-829b-a98b32570292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:23.585807 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893eaed5-3c3b-4fb1-829b-a98b32570292 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:23.598800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1017/2017] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:56:23.602888 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7068e0-53e6-42ec-bf0b-57f643b44ba1 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:23.614486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7068e0-53e6-42ec-bf0b-57f643b44ba1 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:23.625612 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65ec49-6d2f-4e2b-837a-e9e7a884208e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:23.625612 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65ec49-6d2f-4e2b-837a-e9e7a884208e admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:23.625612 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65ec49-6d2f-4e2b-837a-e9e7a884208e admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:23.625612 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65ec49-6d2f-4e2b-837a-e9e7a884208e admin admin] PATH_INFO: `/v3/projects/165d26e917314feab1cc0f428c5cc0c1/users/b643550a47cf4fc8a750b5b4411ded1b/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:23.628107 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e65ec49-6d2f-4e2b-837a-e9e7a884208e admin admin] RBAC: Authorizing `identity:create_grant(project_id=165d26e917314feab1cc0f428c5cc0c1, user_id=b643550a47cf4fc8a750b5b4411ded1b, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:23.630786 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e65ec49-6d2f-4e2b-837a-e9e7a884208e admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:23.641988 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1001/2018] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:56:23 2022] PUT /identity/v3/projects/165d26e917314feab1cc0f428c5cc0c1/users/b643550a47cf4fc8a750b5b4411ded1b/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:56:23.652671 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d39749-51ad-4454-8f34-485d2e6b80d6 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:23.655528 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d39749-51ad-4454-8f34-485d2e6b80d6 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:23.655528 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d39749-51ad-4454-8f34-485d2e6b80d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:23.685570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-90d39749-51ad-4454-8f34-485d2e6b80d6 None None] MFA Rules not processed for user `b643550a47cf4fc8a750b5b4411ded1b`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:56:23.686990 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-90d39749-51ad-4454-8f34-485d2e6b80d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:23.768643 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1018/2019] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:56:23 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:56:23.780042 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65ec49-6d2f-4e2b-837a-e9e7a884208e admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:23.781801 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-2e65ec49-6d2f-4e2b-837a-e9e7a884208e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:23.844645 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65ec49-6d2f-4e2b-837a-e9e7a884208e admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:23.849024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f859c-aeb7-4795-8322-698bc087e208 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:23.849639 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f859c-aeb7-4795-8322-698bc087e208 service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:23.849926 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f859c-aeb7-4795-8322-698bc087e208 service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:23.850198 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f859c-aeb7-4795-8322-698bc087e208 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:23.859609 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f859c-aeb7-4795-8322-698bc087e208 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:23.861094 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f859c-aeb7-4795-8322-698bc087e208 service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:23.871669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1002/2020] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:56:23 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:56:24.013726 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d39749-51ad-4454-8f34-485d2e6b80d6 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:24.060959 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f859c-aeb7-4795-8322-698bc087e208 service cinder] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:24.064772 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-428f859c-aeb7-4795-8322-698bc087e208 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:24.078407 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d39749-51ad-4454-8f34-485d2e6b80d6 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:24.091657 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4887f8-54c2-4b08-861d-d8a7cb143e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:24.092662 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4887f8-54c2-4b08-861d-d8a7cb143e23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:24.093295 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4887f8-54c2-4b08-861d-d8a7cb143e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:24.093295 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4887f8-54c2-4b08-861d-d8a7cb143e23 admin admin] PATH_INFO: `/v3/users/81ca8826c48c4bd698743babd31e0d19` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:24.095214 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4887f8-54c2-4b08-861d-d8a7cb143e23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81ca8826c48c4bd698743babd31e0d19)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:24.098456 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4887f8-54c2-4b08-861d-d8a7cb143e23 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:24.143342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f859c-aeb7-4795-8322-698bc087e208 service cinder] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:24.148973 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd948ef7-f299-4324-9962-cc18345e558c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:24.149593 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd948ef7-f299-4324-9962-cc18345e558c service nova] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:24.149877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd948ef7-f299-4324-9962-cc18345e558c service nova] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:24.150149 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd948ef7-f299-4324-9962-cc18345e558c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:24.152039 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-fd948ef7-f299-4324-9962-cc18345e558c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:24.186957 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:24.244520 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd948ef7-f299-4324-9962-cc18345e558c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:24.246134 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd948ef7-f299-4324-9962-cc18345e558c service nova] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:24.255126 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1003/2021] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:56:24 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:56:24.396960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:24.413411 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-da4887f8-54c2-4b08-861d-d8a7cb143e23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81ca8826c48c4bd698743babd31e0d19'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:24.414111 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-da4887f8-54c2-4b08-861d-d8a7cb143e23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81ca8826c48c4bd698743babd31e0d19'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:24.493307 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-da4887f8-54c2-4b08-861d-d8a7cb143e23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81ca8826c48c4bd698743babd31e0d19'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:24.506217 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-da4887f8-54c2-4b08-861d-d8a7cb143e23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81ca8826c48c4bd698743babd31e0d19'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:24.506903 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-da4887f8-54c2-4b08-861d-d8a7cb143e23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81ca8826c48c4bd698743babd31e0d19'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:24.519671 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1019/2022] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:56:24 2022] DELETE /identity/v3/users/81ca8826c48c4bd698743babd31e0d19 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:56:24.524393 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2a7d1-a7a9-4ea9-a63a-f915e30e06e4 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:24.524682 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2a7d1-a7a9-4ea9-a63a-f915e30e06e4 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:24.524986 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc2a7d1-a7a9-4ea9-a63a-f915e30e06e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:24.544135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-0cc2a7d1-a7a9-4ea9-a63a-f915e30e06e4 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:56:24.545334 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-0cc2a7d1-a7a9-4ea9-a63a-f915e30e06e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:24.606445 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1004/2023] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:56:24.616346 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4887f8-54c2-4b08-861d-d8a7cb143e23 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:24.618324 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-da4887f8-54c2-4b08-861d-d8a7cb143e23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:24.677029 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4887f8-54c2-4b08-861d-d8a7cb143e23 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:24.680867 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114195c-793a-46fa-8ccd-961b03df3707 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:24.681487 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114195c-793a-46fa-8ccd-961b03df3707 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:24.684479 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114195c-793a-46fa-8ccd-961b03df3707 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:24.684479 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114195c-793a-46fa-8ccd-961b03df3707 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:24.691719 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114195c-793a-46fa-8ccd-961b03df3707 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:24.693237 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114195c-793a-46fa-8ccd-961b03df3707 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:24.703173 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1020/2024] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:56:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:56:25.120068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc2a7d1-a7a9-4ea9-a63a-f915e30e06e4 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:25.121492 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-0cc2a7d1-a7a9-4ea9-a63a-f915e30e06e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:25.135256 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc2a7d1-a7a9-4ea9-a63a-f915e30e06e4 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:25.140084 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb39e51-fe93-43b8-99cf-f6157c7f1323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:25.140739 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb39e51-fe93-43b8-99cf-f6157c7f1323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:25.141021 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb39e51-fe93-43b8-99cf-f6157c7f1323 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:25.141299 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb39e51-fe93-43b8-99cf-f6157c7f1323 admin admin] PATH_INFO: `/v3/projects/383031080dc540529bc1e8f4e190a347` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:25.143755 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb39e51-fe93-43b8-99cf-f6157c7f1323 admin admin] RBAC: Authorizing `identity:delete_project(project_id=383031080dc540529bc1e8f4e190a347)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:25.145669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb39e51-fe93-43b8-99cf-f6157c7f1323 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:25.234824 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-0cb39e51-fe93-43b8-99cf-f6157c7f1323 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '383031080dc540529bc1e8f4e190a347', 'request_id': 'req-0cb39e51-fe93-43b8-99cf-f6157c7f1323'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:25.301042 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-0cb39e51-fe93-43b8-99cf-f6157c7f1323 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '383031080dc540529bc1e8f4e190a347', 'request_id': 'req-0cb39e51-fe93-43b8-99cf-f6157c7f1323'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:25.306859 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-0cb39e51-fe93-43b8-99cf-f6157c7f1323 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '383031080dc540529bc1e8f4e190a347', 'request_id': 'req-0cb39e51-fe93-43b8-99cf-f6157c7f1323'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:25.319401 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-0cb39e51-fe93-43b8-99cf-f6157c7f1323 admin admin] The token cache is being invalidate because project 383031080dc540529bc1e8f4e190a347 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:56:25.321542 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-0cb39e51-fe93-43b8-99cf-f6157c7f1323 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:25.321542 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-0cb39e51-fe93-43b8-99cf-f6157c7f1323 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:25.321894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1005/2025] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:56:25 2022] DELETE /identity/v3/projects/383031080dc540529bc1e8f4e190a347 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:56:25.325865 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114195c-793a-46fa-8ccd-961b03df3707 service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:25.335727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114195c-793a-46fa-8ccd-961b03df3707 service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:25.392344 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4ae62-529f-440e-82d0-93ccab30f0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:25.393013 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4ae62-529f-440e-82d0-93ccab30f0cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:25.393294 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4ae62-529f-440e-82d0-93ccab30f0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:25.393568 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4ae62-529f-440e-82d0-93ccab30f0cf admin admin] PATH_INFO: `/v3/users/b643550a47cf4fc8a750b5b4411ded1b` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:25.394695 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4ae62-529f-440e-82d0-93ccab30f0cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b643550a47cf4fc8a750b5b4411ded1b)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:25.396645 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4ae62-529f-440e-82d0-93ccab30f0cf admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:25.586585 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2ea4ae62-529f-440e-82d0-93ccab30f0cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b643550a47cf4fc8a750b5b4411ded1b'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:25.587236 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2ea4ae62-529f-440e-82d0-93ccab30f0cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b643550a47cf4fc8a750b5b4411ded1b'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:25.590915 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:25.676281 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2ea4ae62-529f-440e-82d0-93ccab30f0cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b643550a47cf4fc8a750b5b4411ded1b'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:25.688765 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2ea4ae62-529f-440e-82d0-93ccab30f0cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b643550a47cf4fc8a750b5b4411ded1b'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:25.689426 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2ea4ae62-529f-440e-82d0-93ccab30f0cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b643550a47cf4fc8a750b5b4411ded1b'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:25.701908 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1021/2026] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:56:25 2022] DELETE /identity/v3/users/b643550a47cf4fc8a750b5b4411ded1b => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:56:26.091589 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb39e51-fe93-43b8-99cf-f6157c7f1323 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:26.093211 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-0cb39e51-fe93-43b8-99cf-f6157c7f1323 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:26.106505 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb39e51-fe93-43b8-99cf-f6157c7f1323 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:26.154632 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:26.164505 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052dc70-9fba-4991-b066-055424d32536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:26.165142 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052dc70-9fba-4991-b066-055424d32536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:26.165423 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052dc70-9fba-4991-b066-055424d32536 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:26.165696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052dc70-9fba-4991-b066-055424d32536 admin admin] PATH_INFO: `/v3/projects/165d26e917314feab1cc0f428c5cc0c1` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:26.166813 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4052dc70-9fba-4991-b066-055424d32536 admin admin] RBAC: Authorizing `identity:delete_project(project_id=165d26e917314feab1cc0f428c5cc0c1)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:26.168754 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4052dc70-9fba-4991-b066-055424d32536 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:26.315660 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-4052dc70-9fba-4991-b066-055424d32536 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '165d26e917314feab1cc0f428c5cc0c1', 'request_id': 'req-4052dc70-9fba-4991-b066-055424d32536'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:26.409916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-4052dc70-9fba-4991-b066-055424d32536 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '165d26e917314feab1cc0f428c5cc0c1', 'request_id': 'req-4052dc70-9fba-4991-b066-055424d32536'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:26.416243 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-4052dc70-9fba-4991-b066-055424d32536 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '165d26e917314feab1cc0f428c5cc0c1', 'request_id': 'req-4052dc70-9fba-4991-b066-055424d32536'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:26.428418 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-4052dc70-9fba-4991-b066-055424d32536 admin admin] The token cache is being invalidate because project 165d26e917314feab1cc0f428c5cc0c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:56:26.428788 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-4052dc70-9fba-4991-b066-055424d32536 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:26.429438 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-4052dc70-9fba-4991-b066-055424d32536 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:26.430919 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1006/2027] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:56:26 2022] DELETE /identity/v3/projects/165d26e917314feab1cc0f428c5cc0c1 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:56:26.435789 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf199d9-b0c3-461d-abca-05657278fecd None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:26.436229 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf199d9-b0c3-461d-abca-05657278fecd None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:26.436352 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf199d9-b0c3-461d-abca-05657278fecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:26.457039 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-5bf199d9-b0c3-461d-abca-05657278fecd None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:56:26.458218 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-5bf199d9-b0c3-461d-abca-05657278fecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:26.517015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1022/2028] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:56:26.526818 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13228014-0eed-47b7-9a4f-7542f82beeb4 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:26.527100 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13228014-0eed-47b7-9a4f-7542f82beeb4 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:26.527378 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13228014-0eed-47b7-9a4f-7542f82beeb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:26.547728 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-13228014-0eed-47b7-9a4f-7542f82beeb4 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:56:26.549036 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-13228014-0eed-47b7-9a4f-7542f82beeb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:26.557336 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1007/2029] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:56:26.561559 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf199d9-b0c3-461d-abca-05657278fecd None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:26.571838 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf199d9-b0c3-461d-abca-05657278fecd None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:26.576306 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f0db0-3a84-4349-b92d-f50b5acc915b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:26.576905 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f0db0-3a84-4349-b92d-f50b5acc915b admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:26.577269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f0db0-3a84-4349-b92d-f50b5acc915b admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:26.577465 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f0db0-3a84-4349-b92d-f50b5acc915b admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:26.578108 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f0db0-3a84-4349-b92d-f50b5acc915b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:56:26.578416 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f0db0-3a84-4349-b92d-f50b5acc915b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:26.580038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f0db0-3a84-4349-b92d-f50b5acc915b admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:26.592587 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1023/2030] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:56:26.596773 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13228014-0eed-47b7-9a4f-7542f82beeb4 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:26.606279 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13228014-0eed-47b7-9a4f-7542f82beeb4 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:26.611022 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d554a-7ef5-4b77-b8f0-507add873e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:26.611650 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d554a-7ef5-4b77-b8f0-507add873e87 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:26.611979 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d554a-7ef5-4b77-b8f0-507add873e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:26.617092 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d554a-7ef5-4b77-b8f0-507add873e87 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:26.617092 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d554a-7ef5-4b77-b8f0-507add873e87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:26.617092 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d554a-7ef5-4b77-b8f0-507add873e87 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:26.755350 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1008/2031] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:56:26 2022] POST /identity/v3/projects => generated 367 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:56:26.759465 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f0db0-3a84-4349-b92d-f50b5acc915b admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:26.772151 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f0db0-3a84-4349-b92d-f50b5acc915b admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:26.830629 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e79d03-e999-4a70-bbf5-52f9d2670f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:26.831170 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e79d03-e999-4a70-bbf5-52f9d2670f5d admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:26.831455 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e79d03-e999-4a70-bbf5-52f9d2670f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:26.831727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e79d03-e999-4a70-bbf5-52f9d2670f5d admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:26.832386 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e79d03-e999-4a70-bbf5-52f9d2670f5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:26.834600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e79d03-e999-4a70-bbf5-52f9d2670f5d admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:26.945987 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1024/2032] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:56:26 2022] POST /identity/v3/users => generated 284 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:56:26.950331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d554a-7ef5-4b77-b8f0-507add873e87 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:26.961636 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d554a-7ef5-4b77-b8f0-507add873e87 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:26.966730 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd78cfb-0114-49de-b53c-5d77ef7fe194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:26.967344 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd78cfb-0114-49de-b53c-5d77ef7fe194 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:26.967665 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd78cfb-0114-49de-b53c-5d77ef7fe194 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:26.967917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd78cfb-0114-49de-b53c-5d77ef7fe194 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:26.968601 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd78cfb-0114-49de-b53c-5d77ef7fe194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:26.970135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd78cfb-0114-49de-b53c-5d77ef7fe194 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:26.983227 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1009/2033] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:56:26.984447 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e79d03-e999-4a70-bbf5-52f9d2670f5d admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:26.995144 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e79d03-e999-4a70-bbf5-52f9d2670f5d admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:26.999878 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2762ad1-41b5-4d55-9c54-4ac77cdf35a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:27.000555 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2762ad1-41b5-4d55-9c54-4ac77cdf35a1 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:27.000903 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2762ad1-41b5-4d55-9c54-4ac77cdf35a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:27.001245 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2762ad1-41b5-4d55-9c54-4ac77cdf35a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:27.001986 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2762ad1-41b5-4d55-9c54-4ac77cdf35a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:27.004299 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2762ad1-41b5-4d55-9c54-4ac77cdf35a1 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:27.014530 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1025/2034] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:56:27.018701 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd78cfb-0114-49de-b53c-5d77ef7fe194 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:27.028867 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd78cfb-0114-49de-b53c-5d77ef7fe194 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:27.033726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3fbff-0e1b-4e8a-9e1c-3f26eca5f821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:27.034406 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3fbff-0e1b-4e8a-9e1c-3f26eca5f821 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:27.034690 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3fbff-0e1b-4e8a-9e1c-3f26eca5f821 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:27.035453 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3fbff-0e1b-4e8a-9e1c-3f26eca5f821 admin admin] PATH_INFO: `/v3/projects/abdb1f0de7ae4756a2c49b2c197c21e8/users/36144d5a70404ecb8445cfc1a61f4f1a/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:27.051588 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d3fbff-0e1b-4e8a-9e1c-3f26eca5f821 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abdb1f0de7ae4756a2c49b2c197c21e8, user_id=36144d5a70404ecb8445cfc1a61f4f1a, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:27.054260 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d3fbff-0e1b-4e8a-9e1c-3f26eca5f821 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:27.147641 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1010/2035] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:56:27 2022] PUT /identity/v3/projects/abdb1f0de7ae4756a2c49b2c197c21e8/users/36144d5a70404ecb8445cfc1a61f4f1a/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:56:27.158025 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40496040-07fa-4d0b-8421-2a0c41239eb3 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:27.158309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40496040-07fa-4d0b-8421-2a0c41239eb3 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:27.159080 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40496040-07fa-4d0b-8421-2a0c41239eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:27.187347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-40496040-07fa-4d0b-8421-2a0c41239eb3 None None] MFA Rules not processed for user `36144d5a70404ecb8445cfc1a61f4f1a`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:56:27.188605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-40496040-07fa-4d0b-8421-2a0c41239eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:27.213688 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:27.254229 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1026/2036] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:56:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:56:27.258735 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3fbff-0e1b-4e8a-9e1c-3f26eca5f821 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:27.319492 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3fbff-0e1b-4e8a-9e1c-3f26eca5f821 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:27.324392 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb20a5-2355-49b9-95a3-522e09ea3ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:27.324999 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb20a5-2355-49b9-95a3-522e09ea3ee5 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:27.325281 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb20a5-2355-49b9-95a3-522e09ea3ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:27.325556 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb20a5-2355-49b9-95a3-522e09ea3ee5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:27.326203 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb20a5-2355-49b9-95a3-522e09ea3ee5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:27.328150 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb20a5-2355-49b9-95a3-522e09ea3ee5 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:27.422901 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1011/2037] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:56:27 2022] POST /identity/v3/projects => generated 369 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:56:27.427631 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40496040-07fa-4d0b-8421-2a0c41239eb3 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:27.438934 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40496040-07fa-4d0b-8421-2a0c41239eb3 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:27.495547 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da17e124-ea21-4a3f-8754-9744f8a7e0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:27.496169 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17e124-ea21-4a3f-8754-9744f8a7e0e2 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:27.496451 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17e124-ea21-4a3f-8754-9744f8a7e0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:27.496725 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17e124-ea21-4a3f-8754-9744f8a7e0e2 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:27.497353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da17e124-ea21-4a3f-8754-9744f8a7e0e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:27.499281 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da17e124-ea21-4a3f-8754-9744f8a7e0e2 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:27.598963 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1027/2038] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:56:27 2022] POST /identity/v3/users => generated 291 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:56:27.603174 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb20a5-2355-49b9-95a3-522e09ea3ee5 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:27.609836 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:27.613633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb20a5-2355-49b9-95a3-522e09ea3ee5 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:27.618275 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5cf7a9-1aee-4ecb-89d2-d7c51a133114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:27.618869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5cf7a9-1aee-4ecb-89d2-d7c51a133114 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:27.619145 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5cf7a9-1aee-4ecb-89d2-d7c51a133114 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:27.619416 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5cf7a9-1aee-4ecb-89d2-d7c51a133114 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:27.620156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5cf7a9-1aee-4ecb-89d2-d7c51a133114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:27.621720 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5cf7a9-1aee-4ecb-89d2-d7c51a133114 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:27.632249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1012/2039] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:56:27.636284 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da17e124-ea21-4a3f-8754-9744f8a7e0e2 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:27.646709 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da17e124-ea21-4a3f-8754-9744f8a7e0e2 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:27.651349 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c63b10-8963-4396-8ed3-c39f250f3bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:27.652035 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c63b10-8963-4396-8ed3-c39f250f3bcd admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:27.652316 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c63b10-8963-4396-8ed3-c39f250f3bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:27.652589 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c63b10-8963-4396-8ed3-c39f250f3bcd admin admin] PATH_INFO: `/v3/projects/0b4a3b4e04c54f9d99b3ea39bb97de62/users/bce53dec35964be6814cb019ec63de66/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:27.667843 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c63b10-8963-4396-8ed3-c39f250f3bcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b4a3b4e04c54f9d99b3ea39bb97de62, user_id=bce53dec35964be6814cb019ec63de66, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:27.670479 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c63b10-8963-4396-8ed3-c39f250f3bcd admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:27.770873 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1028/2040] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:56:27 2022] PUT /identity/v3/projects/0b4a3b4e04c54f9d99b3ea39bb97de62/users/bce53dec35964be6814cb019ec63de66/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:56:27.774799 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5cf7a9-1aee-4ecb-89d2-d7c51a133114 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:27.785203 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5cf7a9-1aee-4ecb-89d2-d7c51a133114 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:27.840276 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22d862-a497-46fd-a929-abfb3a517d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:27.840874 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22d862-a497-46fd-a929-abfb3a517d41 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:27.841151 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22d862-a497-46fd-a929-abfb3a517d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:27.841418 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22d862-a497-46fd-a929-abfb3a517d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:27.842042 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe22d862-a497-46fd-a929-abfb3a517d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:27.843604 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe22d862-a497-46fd-a929-abfb3a517d41 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:27.853125 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1013/2041] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:56:27.857150 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c63b10-8963-4396-8ed3-c39f250f3bcd admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:27.867240 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c63b10-8963-4396-8ed3-c39f250f3bcd admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:27.871828 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8734c-ec30-4fde-b1e6-0c0b09c38dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:27.872470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8734c-ec30-4fde-b1e6-0c0b09c38dd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:27.872771 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8734c-ec30-4fde-b1e6-0c0b09c38dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:27.873025 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8734c-ec30-4fde-b1e6-0c0b09c38dd4 admin admin] PATH_INFO: `/v3/projects/0b4a3b4e04c54f9d99b3ea39bb97de62/users/bce53dec35964be6814cb019ec63de66/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:27.874911 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f8734c-ec30-4fde-b1e6-0c0b09c38dd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b4a3b4e04c54f9d99b3ea39bb97de62, user_id=bce53dec35964be6814cb019ec63de66, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:27.878436 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f8734c-ec30-4fde-b1e6-0c0b09c38dd4 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:27.888359 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1029/2042] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:56:27 2022] PUT /identity/v3/projects/0b4a3b4e04c54f9d99b3ea39bb97de62/users/bce53dec35964be6814cb019ec63de66/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:56:27.898888 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b776094-d91a-4ad7-9c72-9063c55361b3 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:27.899175 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b776094-d91a-4ad7-9c72-9063c55361b3 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:27.899451 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b776094-d91a-4ad7-9c72-9063c55361b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:27.926118 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-6b776094-d91a-4ad7-9c72-9063c55361b3 None None] MFA Rules not processed for user `bce53dec35964be6814cb019ec63de66`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:56:27.927337 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-6b776094-d91a-4ad7-9c72-9063c55361b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:27.993837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1014/2043] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:56:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:56:28.004882 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8734c-ec30-4fde-b1e6-0c0b09c38dd4 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:28.006542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-f9f8734c-ec30-4fde-b1e6-0c0b09c38dd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:28.063354 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8734c-ec30-4fde-b1e6-0c0b09c38dd4 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:28.067478 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c29d79-e4a0-4e79-87d6-cfdc84abc202 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:28.068108 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c29d79-e4a0-4e79-87d6-cfdc84abc202 service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:28.068388 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c29d79-e4a0-4e79-87d6-cfdc84abc202 service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:28.068660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c29d79-e4a0-4e79-87d6-cfdc84abc202 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:28.077459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c29d79-e4a0-4e79-87d6-cfdc84abc202 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:28.078873 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c29d79-e4a0-4e79-87d6-cfdc84abc202 service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:28.089061 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1030/2044] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:56:28 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:56:30.853858 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b776094-d91a-4ad7-9c72-9063c55361b3 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:30.914508 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b776094-d91a-4ad7-9c72-9063c55361b3 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:30.919018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f80d4c8-d219-4598-aa39-d3bae95b30dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:30.919641 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80d4c8-d219-4598-aa39-d3bae95b30dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:30.919951 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80d4c8-d219-4598-aa39-d3bae95b30dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:30.920229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f80d4c8-d219-4598-aa39-d3bae95b30dc admin admin] PATH_INFO: `/v3/users/36144d5a70404ecb8445cfc1a61f4f1a` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:30.921337 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f80d4c8-d219-4598-aa39-d3bae95b30dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=36144d5a70404ecb8445cfc1a61f4f1a)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:30.923238 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f80d4c8-d219-4598-aa39-d3bae95b30dc admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:30.938398 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:31.137881 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-9f80d4c8-d219-4598-aa39-d3bae95b30dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36144d5a70404ecb8445cfc1a61f4f1a'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:31.201681 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-9f80d4c8-d219-4598-aa39-d3bae95b30dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36144d5a70404ecb8445cfc1a61f4f1a'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:31.215541 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-9f80d4c8-d219-4598-aa39-d3bae95b30dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36144d5a70404ecb8445cfc1a61f4f1a'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:31.215541 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-9f80d4c8-d219-4598-aa39-d3bae95b30dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36144d5a70404ecb8445cfc1a61f4f1a'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:31.216091 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-9f80d4c8-d219-4598-aa39-d3bae95b30dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36144d5a70404ecb8445cfc1a61f4f1a'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:31.229246 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1015/2045] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:56:30 2022] DELETE /identity/v3/users/36144d5a70404ecb8445cfc1a61f4f1a => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:56:31.233689 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c46c0-0007-499f-a4d5-7ebc207fdefb None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:31.234148 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c46c0-0007-499f-a4d5-7ebc207fdefb None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:31.234242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c46c0-0007-499f-a4d5-7ebc207fdefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:31.252874 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-7c5c46c0-0007-499f-a4d5-7ebc207fdefb None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:56:31.254050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-7c5c46c0-0007-499f-a4d5-7ebc207fdefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:31.266420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:31.312168 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1031/2046] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:56:31.321846 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f80d4c8-d219-4598-aa39-d3bae95b30dc admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:31.323228 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-9f80d4c8-d219-4598-aa39-d3bae95b30dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:31.379453 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f80d4c8-d219-4598-aa39-d3bae95b30dc admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:31.383086 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aea0aa-a80f-49c1-9903-02f4d438a68b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:31.383696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aea0aa-a80f-49c1-9903-02f4d438a68b service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:31.384012 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aea0aa-a80f-49c1-9903-02f4d438a68b service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:31.384285 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aea0aa-a80f-49c1-9903-02f4d438a68b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:31.393792 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aea0aa-a80f-49c1-9903-02f4d438a68b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:31.395228 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8aea0aa-a80f-49c1-9903-02f4d438a68b service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:31.404519 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1016/2047] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:56:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:56:31.845780 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c46c0-0007-499f-a4d5-7ebc207fdefb None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:31.847423 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-7c5c46c0-0007-499f-a4d5-7ebc207fdefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:31.861124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c46c0-0007-499f-a4d5-7ebc207fdefb None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:31.865779 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2359af-6217-4b28-8317-0fdb989b0bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:31.866418 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2359af-6217-4b28-8317-0fdb989b0bc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:31.866699 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2359af-6217-4b28-8317-0fdb989b0bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:31.866974 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2359af-6217-4b28-8317-0fdb989b0bc6 admin admin] PATH_INFO: `/v3/projects/abdb1f0de7ae4756a2c49b2c197c21e8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:31.868142 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2359af-6217-4b28-8317-0fdb989b0bc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abdb1f0de7ae4756a2c49b2c197c21e8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:31.870049 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2359af-6217-4b28-8317-0fdb989b0bc6 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:31.981409 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-0e2359af-6217-4b28-8317-0fdb989b0bc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abdb1f0de7ae4756a2c49b2c197c21e8', 'request_id': 'req-0e2359af-6217-4b28-8317-0fdb989b0bc6'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:32.050578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-0e2359af-6217-4b28-8317-0fdb989b0bc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abdb1f0de7ae4756a2c49b2c197c21e8', 'request_id': 'req-0e2359af-6217-4b28-8317-0fdb989b0bc6'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:32.056526 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-0e2359af-6217-4b28-8317-0fdb989b0bc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abdb1f0de7ae4756a2c49b2c197c21e8', 'request_id': 'req-0e2359af-6217-4b28-8317-0fdb989b0bc6'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:32.068339 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-0e2359af-6217-4b28-8317-0fdb989b0bc6 admin admin] The token cache is being invalidate because project abdb1f0de7ae4756a2c49b2c197c21e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:56:32.068627 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-0e2359af-6217-4b28-8317-0fdb989b0bc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:32.069255 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-0e2359af-6217-4b28-8317-0fdb989b0bc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:32.070739 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1032/2048] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:56:31 2022] DELETE /identity/v3/projects/abdb1f0de7ae4756a2c49b2c197c21e8 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:56:32.074696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aea0aa-a80f-49c1-9903-02f4d438a68b service neutron] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:32.084511 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aea0aa-a80f-49c1-9903-02f4d438a68b service neutron] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:32.140617 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3054e89-e1af-436b-af69-9e7ff320bb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:32.141244 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3054e89-e1af-436b-af69-9e7ff320bb77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:32.141533 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3054e89-e1af-436b-af69-9e7ff320bb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:32.141807 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3054e89-e1af-436b-af69-9e7ff320bb77 admin admin] PATH_INFO: `/v3/users/bce53dec35964be6814cb019ec63de66` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:32.142965 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3054e89-e1af-436b-af69-9e7ff320bb77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bce53dec35964be6814cb019ec63de66)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:32.144915 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3054e89-e1af-436b-af69-9e7ff320bb77 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:32.355641 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:32.361386 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-b3054e89-e1af-436b-af69-9e7ff320bb77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bce53dec35964be6814cb019ec63de66'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:32.444475 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-b3054e89-e1af-436b-af69-9e7ff320bb77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bce53dec35964be6814cb019ec63de66'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:32.456873 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-b3054e89-e1af-436b-af69-9e7ff320bb77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bce53dec35964be6814cb019ec63de66'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:32.457640 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-b3054e89-e1af-436b-af69-9e7ff320bb77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bce53dec35964be6814cb019ec63de66'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:32.458983 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-b3054e89-e1af-436b-af69-9e7ff320bb77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bce53dec35964be6814cb019ec63de66'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:32.470874 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1017/2049] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:56:32 2022] DELETE /identity/v3/users/bce53dec35964be6814cb019ec63de66 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:56:32.866149 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2359af-6217-4b28-8317-0fdb989b0bc6 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:32.867674 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-0e2359af-6217-4b28-8317-0fdb989b0bc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:32.887029 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2359af-6217-4b28-8317-0fdb989b0bc6 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:32.957151 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ff2f96-6d55-43e4-a62c-6e9be150f381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:32.957792 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff2f96-6d55-43e4-a62c-6e9be150f381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:32.958101 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff2f96-6d55-43e4-a62c-6e9be150f381 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:32.958377 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ff2f96-6d55-43e4-a62c-6e9be150f381 admin admin] PATH_INFO: `/v3/projects/0b4a3b4e04c54f9d99b3ea39bb97de62` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:32.959836 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ff2f96-6d55-43e4-a62c-6e9be150f381 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b4a3b4e04c54f9d99b3ea39bb97de62)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:32.961750 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ff2f96-6d55-43e4-a62c-6e9be150f381 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:33.068424 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:33.071234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-10ff2f96-6d55-43e4-a62c-6e9be150f381 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b4a3b4e04c54f9d99b3ea39bb97de62', 'request_id': 'req-10ff2f96-6d55-43e4-a62c-6e9be150f381'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:33.161837 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-10ff2f96-6d55-43e4-a62c-6e9be150f381 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b4a3b4e04c54f9d99b3ea39bb97de62', 'request_id': 'req-10ff2f96-6d55-43e4-a62c-6e9be150f381'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:33.168047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-10ff2f96-6d55-43e4-a62c-6e9be150f381 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b4a3b4e04c54f9d99b3ea39bb97de62', 'request_id': 'req-10ff2f96-6d55-43e4-a62c-6e9be150f381'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:33.180257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-10ff2f96-6d55-43e4-a62c-6e9be150f381 admin admin] The token cache is being invalidate because project 0b4a3b4e04c54f9d99b3ea39bb97de62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:56:33.180545 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-10ff2f96-6d55-43e4-a62c-6e9be150f381 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:33.181172 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-10ff2f96-6d55-43e4-a62c-6e9be150f381 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:33.182644 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1033/2050] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:56:32 2022] DELETE /identity/v3/projects/0b4a3b4e04c54f9d99b3ea39bb97de62 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:56:33.187719 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae235d2-48e1-4d58-bfd2-a980b3e8ae56 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:33.188210 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae235d2-48e1-4d58-bfd2-a980b3e8ae56 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:33.188306 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae235d2-48e1-4d58-bfd2-a980b3e8ae56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:33.210717 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-7ae235d2-48e1-4d58-bfd2-a980b3e8ae56 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:56:33.212208 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-7ae235d2-48e1-4d58-bfd2-a980b3e8ae56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:33.271327 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1018/2051] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:56:33.279088 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2fd802-79b2-4aca-bc2b-463d9e7ca83d None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:33.279372 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2fd802-79b2-4aca-bc2b-463d9e7ca83d None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:33.279645 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2fd802-79b2-4aca-bc2b-463d9e7ca83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:33.300168 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-cd2fd802-79b2-4aca-bc2b-463d9e7ca83d None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:56:33.301352 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-cd2fd802-79b2-4aca-bc2b-463d9e7ca83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:33.309284 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1034/2052] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:56:33.313397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae235d2-48e1-4d58-bfd2-a980b3e8ae56 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:33.323532 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae235d2-48e1-4d58-bfd2-a980b3e8ae56 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:33.328077 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0ddb0-b549-40f4-a788-42682cca2477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:33.328675 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0ddb0-b549-40f4-a788-42682cca2477 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:33.328954 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0ddb0-b549-40f4-a788-42682cca2477 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:33.329231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0ddb0-b549-40f4-a788-42682cca2477 admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:33.329871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d0ddb0-b549-40f4-a788-42682cca2477 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:56:33.330169 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d0ddb0-b549-40f4-a788-42682cca2477 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:33.332806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d0ddb0-b549-40f4-a788-42682cca2477 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:33.344600 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1019/2053] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:56:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:56:33.348658 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2fd802-79b2-4aca-bc2b-463d9e7ca83d None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:33.358216 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2fd802-79b2-4aca-bc2b-463d9e7ca83d None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:33.363135 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14327a3-1ba4-4c7a-b870-81f47ec666c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:33.363727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14327a3-1ba4-4c7a-b870-81f47ec666c0 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:33.364027 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14327a3-1ba4-4c7a-b870-81f47ec666c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:33.364294 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14327a3-1ba4-4c7a-b870-81f47ec666c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:33.364926 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14327a3-1ba4-4c7a-b870-81f47ec666c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:33.366854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14327a3-1ba4-4c7a-b870-81f47ec666c0 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:33.443142 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1035/2054] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:56:33 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:56:33.447864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0ddb0-b549-40f4-a788-42682cca2477 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:33.457693 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0ddb0-b549-40f4-a788-42682cca2477 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:33.512641 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e23fc68-fa92-4a33-9a1e-a6fd92da7f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:33.513245 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23fc68-fa92-4a33-9a1e-a6fd92da7f9d admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:33.513527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23fc68-fa92-4a33-9a1e-a6fd92da7f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:33.513805 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23fc68-fa92-4a33-9a1e-a6fd92da7f9d admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:33.514441 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e23fc68-fa92-4a33-9a1e-a6fd92da7f9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:33.516406 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e23fc68-fa92-4a33-9a1e-a6fd92da7f9d admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:33.610016 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1020/2055] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:56:33 2022] POST /identity/v3/users => generated 292 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:56:33.614733 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14327a3-1ba4-4c7a-b870-81f47ec666c0 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:33.625114 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14327a3-1ba4-4c7a-b870-81f47ec666c0 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:33.630307 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a00fec-0744-4e9b-8139-984434c63bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:33.630898 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a00fec-0744-4e9b-8139-984434c63bca admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:33.631176 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a00fec-0744-4e9b-8139-984434c63bca admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:33.631473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a00fec-0744-4e9b-8139-984434c63bca admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:33.632144 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a00fec-0744-4e9b-8139-984434c63bca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:33.633716 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a00fec-0744-4e9b-8139-984434c63bca admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:33.643992 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1036/2056] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:56:33.647810 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e23fc68-fa92-4a33-9a1e-a6fd92da7f9d admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:33.657858 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e23fc68-fa92-4a33-9a1e-a6fd92da7f9d admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:33.662517 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0850391-975f-45a3-8df4-217c8f201772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:33.663114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0850391-975f-45a3-8df4-217c8f201772 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:33.663426 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0850391-975f-45a3-8df4-217c8f201772 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:33.663706 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0850391-975f-45a3-8df4-217c8f201772 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:33.664382 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0850391-975f-45a3-8df4-217c8f201772 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:33.665978 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0850391-975f-45a3-8df4-217c8f201772 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:33.676866 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1021/2057] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:56:33.681071 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a00fec-0744-4e9b-8139-984434c63bca admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:33.692219 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a00fec-0744-4e9b-8139-984434c63bca admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:33.696909 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc021fd4-4f4c-4d30-bb87-95a5c8b8839a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:33.697499 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc021fd4-4f4c-4d30-bb87-95a5c8b8839a admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:33.697787 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc021fd4-4f4c-4d30-bb87-95a5c8b8839a admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:33.698080 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc021fd4-4f4c-4d30-bb87-95a5c8b8839a admin admin] PATH_INFO: `/v3/projects/8539c64a60e04bcc99034d3bc98de5bc/users/e6ac6260415441efb1e9edad30968247/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:33.713037 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc021fd4-4f4c-4d30-bb87-95a5c8b8839a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8539c64a60e04bcc99034d3bc98de5bc, user_id=e6ac6260415441efb1e9edad30968247, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:33.715663 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc021fd4-4f4c-4d30-bb87-95a5c8b8839a admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:33.792518 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1037/2058] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:56:33 2022] PUT /identity/v3/projects/8539c64a60e04bcc99034d3bc98de5bc/users/e6ac6260415441efb1e9edad30968247/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:56:33.803292 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1865b-5ccb-482a-b426-9dc54d415387 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:33.803659 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1865b-5ccb-482a-b426-9dc54d415387 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:33.803962 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa1865b-5ccb-482a-b426-9dc54d415387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:33.831588 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-0aa1865b-5ccb-482a-b426-9dc54d415387 None None] MFA Rules not processed for user `e6ac6260415441efb1e9edad30968247`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:56:33.832820 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-0aa1865b-5ccb-482a-b426-9dc54d415387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:33.842020 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:33.895696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1022/2059] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:56:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:56:33.900192 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc021fd4-4f4c-4d30-bb87-95a5c8b8839a admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:33.960982 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc021fd4-4f4c-4d30-bb87-95a5c8b8839a admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:33.965639 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60dd895-9ced-4484-b83e-e672dd2a055e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:33.966248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60dd895-9ced-4484-b83e-e672dd2a055e admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:33.966537 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60dd895-9ced-4484-b83e-e672dd2a055e admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:33.966814 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60dd895-9ced-4484-b83e-e672dd2a055e admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:33.967449 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60dd895-9ced-4484-b83e-e672dd2a055e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:33.969392 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60dd895-9ced-4484-b83e-e672dd2a055e admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:34.045569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1038/2060] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:56:33 2022] POST /identity/v3/projects => generated 385 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:56:34.050505 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1865b-5ccb-482a-b426-9dc54d415387 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:34.058063 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa1865b-5ccb-482a-b426-9dc54d415387 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:34.115622 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d789b0-77fa-4806-801a-587a16f047f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:34.116278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d789b0-77fa-4806-801a-587a16f047f5 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:34.116562 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d789b0-77fa-4806-801a-587a16f047f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:34.116843 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d789b0-77fa-4806-801a-587a16f047f5 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:34.117491 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d789b0-77fa-4806-801a-587a16f047f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:34.119407 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d789b0-77fa-4806-801a-587a16f047f5 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:34.201817 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1023/2061] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:56:34 2022] POST /identity/v3/users => generated 299 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:56:34.206232 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60dd895-9ced-4484-b83e-e672dd2a055e admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:34.216934 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60dd895-9ced-4484-b83e-e672dd2a055e admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:34.222456 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95045585-5f5d-4051-b56d-3db092b73a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:34.223137 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95045585-5f5d-4051-b56d-3db092b73a91 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:34.223417 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95045585-5f5d-4051-b56d-3db092b73a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:34.223686 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95045585-5f5d-4051-b56d-3db092b73a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:34.224363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95045585-5f5d-4051-b56d-3db092b73a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:34.225932 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95045585-5f5d-4051-b56d-3db092b73a91 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:34.236505 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1039/2062] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:56:34.240497 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d789b0-77fa-4806-801a-587a16f047f5 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:34.250245 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d789b0-77fa-4806-801a-587a16f047f5 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:34.254743 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc9c14-ea8d-4a4d-91de-67fbffb7aba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:34.255384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc9c14-ea8d-4a4d-91de-67fbffb7aba6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:34.255663 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc9c14-ea8d-4a4d-91de-67fbffb7aba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:34.255968 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc9c14-ea8d-4a4d-91de-67fbffb7aba6 admin admin] PATH_INFO: `/v3/projects/fe76b8c6f5b94b54ab1c14c76561a0f9/users/62e11f5f605b41cd8a0372df95d45a55/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:34.270604 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc9c14-ea8d-4a4d-91de-67fbffb7aba6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe76b8c6f5b94b54ab1c14c76561a0f9, user_id=62e11f5f605b41cd8a0372df95d45a55, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:34.273240 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc9c14-ea8d-4a4d-91de-67fbffb7aba6 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:34.355370 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1024/2063] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:56:34 2022] PUT /identity/v3/projects/fe76b8c6f5b94b54ab1c14c76561a0f9/users/62e11f5f605b41cd8a0372df95d45a55/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:56:34.359193 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95045585-5f5d-4051-b56d-3db092b73a91 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:34.369552 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95045585-5f5d-4051-b56d-3db092b73a91 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:34.410960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:34.425229 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447f9e18-d7ef-4b88-91fd-c43a68480d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:34.425834 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f9e18-d7ef-4b88-91fd-c43a68480d1f admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:34.426111 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f9e18-d7ef-4b88-91fd-c43a68480d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:34.426379 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447f9e18-d7ef-4b88-91fd-c43a68480d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:34.427003 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f9e18-d7ef-4b88-91fd-c43a68480d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:34.428601 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447f9e18-d7ef-4b88-91fd-c43a68480d1f admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:34.437996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1040/2064] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:56:34.442150 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc9c14-ea8d-4a4d-91de-67fbffb7aba6 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:34.452490 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc9c14-ea8d-4a4d-91de-67fbffb7aba6 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:34.457860 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63741cfd-69e1-44f3-96cd-cf40a483d0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:34.458550 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63741cfd-69e1-44f3-96cd-cf40a483d0a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:34.459089 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63741cfd-69e1-44f3-96cd-cf40a483d0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:34.459365 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63741cfd-69e1-44f3-96cd-cf40a483d0a2 admin admin] PATH_INFO: `/v3/projects/fe76b8c6f5b94b54ab1c14c76561a0f9/users/62e11f5f605b41cd8a0372df95d45a55/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:34.461357 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63741cfd-69e1-44f3-96cd-cf40a483d0a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe76b8c6f5b94b54ab1c14c76561a0f9, user_id=62e11f5f605b41cd8a0372df95d45a55, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:34.464026 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63741cfd-69e1-44f3-96cd-cf40a483d0a2 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:34.474988 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1025/2065] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:56:34 2022] PUT /identity/v3/projects/fe76b8c6f5b94b54ab1c14c76561a0f9/users/62e11f5f605b41cd8a0372df95d45a55/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:56:34.485971 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1994f1-3b34-4520-b15b-f658ca0277a8 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:34.489392 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1994f1-3b34-4520-b15b-f658ca0277a8 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:34.489392 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1994f1-3b34-4520-b15b-f658ca0277a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:34.513477 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-cd1994f1-3b34-4520-b15b-f658ca0277a8 None None] MFA Rules not processed for user `62e11f5f605b41cd8a0372df95d45a55`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:56:34.514688 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-cd1994f1-3b34-4520-b15b-f658ca0277a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:34.583503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1041/2066] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:56:34 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:56:34.594800 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63741cfd-69e1-44f3-96cd-cf40a483d0a2 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:34.596536 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-63741cfd-69e1-44f3-96cd-cf40a483d0a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:34.653519 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63741cfd-69e1-44f3-96cd-cf40a483d0a2 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:34.657090 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe17562-422f-48a1-8929-fd7578d7978c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:34.657684 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe17562-422f-48a1-8929-fd7578d7978c service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:34.657964 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe17562-422f-48a1-8929-fd7578d7978c service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:34.658237 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe17562-422f-48a1-8929-fd7578d7978c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:34.667174 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe17562-422f-48a1-8929-fd7578d7978c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:34.668608 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe17562-422f-48a1-8929-fd7578d7978c service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:34.677944 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1026/2067] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:56:34 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:56:34.707288 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1994f1-3b34-4520-b15b-f658ca0277a8 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:34.767417 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1994f1-3b34-4520-b15b-f658ca0277a8 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:34.772531 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb5c00-90cd-41e5-8059-05ad625e83bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:34.773152 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb5c00-90cd-41e5-8059-05ad625e83bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:34.773434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb5c00-90cd-41e5-8059-05ad625e83bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:34.773709 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb5c00-90cd-41e5-8059-05ad625e83bf admin admin] PATH_INFO: `/v3/users/e6ac6260415441efb1e9edad30968247` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:34.774842 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb5c00-90cd-41e5-8059-05ad625e83bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6ac6260415441efb1e9edad30968247)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:34.776812 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb5c00-90cd-41e5-8059-05ad625e83bf admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:34.966870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-94cb5c00-90cd-41e5-8059-05ad625e83bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6ac6260415441efb1e9edad30968247'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:34.967544 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-94cb5c00-90cd-41e5-8059-05ad625e83bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6ac6260415441efb1e9edad30968247'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:35.044433 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-94cb5c00-90cd-41e5-8059-05ad625e83bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6ac6260415441efb1e9edad30968247'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:35.057155 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-94cb5c00-90cd-41e5-8059-05ad625e83bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6ac6260415441efb1e9edad30968247'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:35.057900 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-94cb5c00-90cd-41e5-8059-05ad625e83bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6ac6260415441efb1e9edad30968247'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:35.059105 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:35.071145 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1042/2068] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:56:34 2022] DELETE /identity/v3/users/e6ac6260415441efb1e9edad30968247 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:56:35.075847 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec01d4-39fa-45fb-8fb7-0aa5b22a5455 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:35.076135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec01d4-39fa-45fb-8fb7-0aa5b22a5455 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:35.076411 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec01d4-39fa-45fb-8fb7-0aa5b22a5455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:35.089670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:35.095465 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-3bec01d4-39fa-45fb-8fb7-0aa5b22a5455 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:56:35.096787 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-3bec01d4-39fa-45fb-8fb7-0aa5b22a5455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:35.153607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1027/2069] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:56:35.163315 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb5c00-90cd-41e5-8059-05ad625e83bf admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:35.166232 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-94cb5c00-90cd-41e5-8059-05ad625e83bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:35.223866 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb5c00-90cd-41e5-8059-05ad625e83bf admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:35.226698 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1635aa6-a3dd-4c8e-814e-58fb0a7c55c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:35.227315 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1635aa6-a3dd-4c8e-814e-58fb0a7c55c3 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:35.227601 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1635aa6-a3dd-4c8e-814e-58fb0a7c55c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:35.227894 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1635aa6-a3dd-4c8e-814e-58fb0a7c55c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:35.237533 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1635aa6-a3dd-4c8e-814e-58fb0a7c55c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:35.238954 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1635aa6-a3dd-4c8e-814e-58fb0a7c55c3 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:35.248531 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1043/2070] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:56:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:56:35.637247 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec01d4-39fa-45fb-8fb7-0aa5b22a5455 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:35.638770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-3bec01d4-39fa-45fb-8fb7-0aa5b22a5455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:35.650822 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec01d4-39fa-45fb-8fb7-0aa5b22a5455 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:35.655524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0ff40-6134-4b52-8cc0-14dc9c258275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:35.656186 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0ff40-6134-4b52-8cc0-14dc9c258275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:35.656464 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0ff40-6134-4b52-8cc0-14dc9c258275 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:35.656736 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0ff40-6134-4b52-8cc0-14dc9c258275 admin admin] PATH_INFO: `/v3/projects/8539c64a60e04bcc99034d3bc98de5bc` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:35.657877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d0ff40-6134-4b52-8cc0-14dc9c258275 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8539c64a60e04bcc99034d3bc98de5bc)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:35.660310 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d0ff40-6134-4b52-8cc0-14dc9c258275 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:35.773792 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-39d0ff40-6134-4b52-8cc0-14dc9c258275 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8539c64a60e04bcc99034d3bc98de5bc', 'request_id': 'req-39d0ff40-6134-4b52-8cc0-14dc9c258275'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:35.841109 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-39d0ff40-6134-4b52-8cc0-14dc9c258275 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8539c64a60e04bcc99034d3bc98de5bc', 'request_id': 'req-39d0ff40-6134-4b52-8cc0-14dc9c258275'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:35.847482 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-39d0ff40-6134-4b52-8cc0-14dc9c258275 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8539c64a60e04bcc99034d3bc98de5bc', 'request_id': 'req-39d0ff40-6134-4b52-8cc0-14dc9c258275'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:35.858655 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-39d0ff40-6134-4b52-8cc0-14dc9c258275 admin admin] The token cache is being invalidate because project 8539c64a60e04bcc99034d3bc98de5bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:56:35.858934 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-39d0ff40-6134-4b52-8cc0-14dc9c258275 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:35.859574 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-39d0ff40-6134-4b52-8cc0-14dc9c258275 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:35.861068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1028/2071] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:56:35 2022] DELETE /identity/v3/projects/8539c64a60e04bcc99034d3bc98de5bc => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:56:35.864993 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1635aa6-a3dd-4c8e-814e-58fb0a7c55c3 service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:35.874539 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1635aa6-a3dd-4c8e-814e-58fb0a7c55c3 service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:35.931094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606480c7-ef61-4c2c-b3c5-26dc5bfcf5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:35.931631 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606480c7-ef61-4c2c-b3c5-26dc5bfcf5ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:35.931947 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606480c7-ef61-4c2c-b3c5-26dc5bfcf5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:35.932223 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606480c7-ef61-4c2c-b3c5-26dc5bfcf5ce admin admin] PATH_INFO: `/v3/users/62e11f5f605b41cd8a0372df95d45a55` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:35.933335 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606480c7-ef61-4c2c-b3c5-26dc5bfcf5ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=62e11f5f605b41cd8a0372df95d45a55)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:35.935384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606480c7-ef61-4c2c-b3c5-26dc5bfcf5ce admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:36.092229 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-606480c7-ef61-4c2c-b3c5-26dc5bfcf5ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62e11f5f605b41cd8a0372df95d45a55'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:36.093009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-606480c7-ef61-4c2c-b3c5-26dc5bfcf5ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62e11f5f605b41cd8a0372df95d45a55'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:36.164459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-606480c7-ef61-4c2c-b3c5-26dc5bfcf5ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62e11f5f605b41cd8a0372df95d45a55'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:36.177079 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-606480c7-ef61-4c2c-b3c5-26dc5bfcf5ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62e11f5f605b41cd8a0372df95d45a55'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:36.177708 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-606480c7-ef61-4c2c-b3c5-26dc5bfcf5ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62e11f5f605b41cd8a0372df95d45a55'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:36.192890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1044/2072] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:56:35 2022] DELETE /identity/v3/users/62e11f5f605b41cd8a0372df95d45a55 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:56:36.654780 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0ff40-6134-4b52-8cc0-14dc9c258275 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:36.656243 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-39d0ff40-6134-4b52-8cc0-14dc9c258275 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:36.669342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0ff40-6134-4b52-8cc0-14dc9c258275 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:36.730190 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc81adf-f4bb-44b7-9200-f266a6e74a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:36.730825 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc81adf-f4bb-44b7-9200-f266a6e74a7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:36.731105 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc81adf-f4bb-44b7-9200-f266a6e74a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:36.731377 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc81adf-f4bb-44b7-9200-f266a6e74a7b admin admin] PATH_INFO: `/v3/projects/fe76b8c6f5b94b54ab1c14c76561a0f9` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:36.732537 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc81adf-f4bb-44b7-9200-f266a6e74a7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe76b8c6f5b94b54ab1c14c76561a0f9)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:36.734429 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc81adf-f4bb-44b7-9200-f266a6e74a7b admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:36.841329 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:36.852806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-0fc81adf-f4bb-44b7-9200-f266a6e74a7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe76b8c6f5b94b54ab1c14c76561a0f9', 'request_id': 'req-0fc81adf-f4bb-44b7-9200-f266a6e74a7b'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:36.921707 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-0fc81adf-f4bb-44b7-9200-f266a6e74a7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe76b8c6f5b94b54ab1c14c76561a0f9', 'request_id': 'req-0fc81adf-f4bb-44b7-9200-f266a6e74a7b'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:36.927288 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-0fc81adf-f4bb-44b7-9200-f266a6e74a7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe76b8c6f5b94b54ab1c14c76561a0f9', 'request_id': 'req-0fc81adf-f4bb-44b7-9200-f266a6e74a7b'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:36.939418 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-0fc81adf-f4bb-44b7-9200-f266a6e74a7b admin admin] The token cache is being invalidate because project fe76b8c6f5b94b54ab1c14c76561a0f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:56:36.940229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-0fc81adf-f4bb-44b7-9200-f266a6e74a7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:36.949161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-0fc81adf-f4bb-44b7-9200-f266a6e74a7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:36.949161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1029/2073] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:56:36 2022] DELETE /identity/v3/projects/fe76b8c6f5b94b54ab1c14c76561a0f9 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:56:36.949424 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074f280-71cd-4188-83b4-18966792afc9 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:36.949424 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074f280-71cd-4188-83b4-18966792afc9 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:36.949424 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074f280-71cd-4188-83b4-18966792afc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:36.962183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:36.968166 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-6074f280-71cd-4188-83b4-18966792afc9 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:56:36.969364 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6074f280-71cd-4188-83b4-18966792afc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:37.027163 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1045/2074] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:56:37.034883 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcddc21-e7cc-4c04-8ea6-0ce60a5dae13 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:37.035202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcddc21-e7cc-4c04-8ea6-0ce60a5dae13 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:37.035483 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcddc21-e7cc-4c04-8ea6-0ce60a5dae13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:37.055015 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-1bcddc21-e7cc-4c04-8ea6-0ce60a5dae13 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:56:37.056208 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-1bcddc21-e7cc-4c04-8ea6-0ce60a5dae13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:37.064070 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1030/2075] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:56:37.068659 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074f280-71cd-4188-83b4-18966792afc9 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:37.078229 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074f280-71cd-4188-83b4-18966792afc9 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:37.083441 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfd035-df09-4931-b49f-56065a2d43a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:37.084342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfd035-df09-4931-b49f-56065a2d43a7 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:37.084620 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfd035-df09-4931-b49f-56065a2d43a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:37.084893 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfd035-df09-4931-b49f-56065a2d43a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:37.085546 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabfd035-df09-4931-b49f-56065a2d43a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:56:37.085851 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabfd035-df09-4931-b49f-56065a2d43a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:37.087427 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabfd035-df09-4931-b49f-56065a2d43a7 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:37.098893 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1046/2076] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:56:37.103169 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcddc21-e7cc-4c04-8ea6-0ce60a5dae13 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:37.112174 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcddc21-e7cc-4c04-8ea6-0ce60a5dae13 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:37.117030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f5df9-1ecc-4711-9dda-b584a9f3311e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:37.117622 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f5df9-1ecc-4711-9dda-b584a9f3311e admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:37.117900 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f5df9-1ecc-4711-9dda-b584a9f3311e admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:37.118234 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4f5df9-1ecc-4711-9dda-b584a9f3311e admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:37.118873 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4f5df9-1ecc-4711-9dda-b584a9f3311e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:37.120853 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4f5df9-1ecc-4711-9dda-b584a9f3311e admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:37.223002 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1031/2077] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:56:37 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:56:37.227015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfd035-df09-4931-b49f-56065a2d43a7 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:37.237470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfd035-df09-4931-b49f-56065a2d43a7 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:37.292716 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89647147-161a-4115-834f-b365d345423b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:37.293276 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89647147-161a-4115-834f-b365d345423b admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:37.293556 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89647147-161a-4115-834f-b365d345423b admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:37.293828 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89647147-161a-4115-834f-b365d345423b admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:37.294471 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89647147-161a-4115-834f-b365d345423b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:37.296419 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89647147-161a-4115-834f-b365d345423b admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:37.373001 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1047/2078] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:56:37 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:56:37.374205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f5df9-1ecc-4711-9dda-b584a9f3311e admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:37.384923 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4f5df9-1ecc-4711-9dda-b584a9f3311e admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:37.389781 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9e37b-fd55-45af-a301-9967cc246215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:37.391226 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9e37b-fd55-45af-a301-9967cc246215 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:37.391226 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9e37b-fd55-45af-a301-9967cc246215 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:37.391226 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9e37b-fd55-45af-a301-9967cc246215 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:37.391539 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df9e37b-fd55-45af-a301-9967cc246215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:37.393173 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df9e37b-fd55-45af-a301-9967cc246215 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:37.403645 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1032/2079] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:56:37.407009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89647147-161a-4115-834f-b365d345423b admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:37.417011 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89647147-161a-4115-834f-b365d345423b admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:37.421576 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa7945-52b7-40f2-ab60-a2a711a8c320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:37.422165 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa7945-52b7-40f2-ab60-a2a711a8c320 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:37.422447 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa7945-52b7-40f2-ab60-a2a711a8c320 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:37.422718 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa7945-52b7-40f2-ab60-a2a711a8c320 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:37.423347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efa7945-52b7-40f2-ab60-a2a711a8c320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:37.425450 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efa7945-52b7-40f2-ab60-a2a711a8c320 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:37.435861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1048/2080] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:56:37.440785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9e37b-fd55-45af-a301-9967cc246215 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:37.450636 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9e37b-fd55-45af-a301-9967cc246215 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:37.455362 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67ed00-4150-45bc-9d34-d4ac0743e872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:37.456146 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67ed00-4150-45bc-9d34-d4ac0743e872 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:37.456424 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67ed00-4150-45bc-9d34-d4ac0743e872 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:37.456694 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67ed00-4150-45bc-9d34-d4ac0743e872 admin admin] PATH_INFO: `/v3/projects/ffa18779c9f1446f9f82cb3f6abac921/users/f2310da421c64dada405f3a09b550bd7/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:37.471434 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67ed00-4150-45bc-9d34-d4ac0743e872 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffa18779c9f1446f9f82cb3f6abac921, user_id=f2310da421c64dada405f3a09b550bd7, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:37.474151 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67ed00-4150-45bc-9d34-d4ac0743e872 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:37.544648 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1033/2081] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:56:37 2022] PUT /identity/v3/projects/ffa18779c9f1446f9f82cb3f6abac921/users/f2310da421c64dada405f3a09b550bd7/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:56:37.555654 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a0243-b7b5-4c9b-ae00-b22d58a79910 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:37.555982 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a0243-b7b5-4c9b-ae00-b22d58a79910 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:37.556257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a0243-b7b5-4c9b-ae00-b22d58a79910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:37.583573 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-6d0a0243-b7b5-4c9b-ae00-b22d58a79910 None None] MFA Rules not processed for user `f2310da421c64dada405f3a09b550bd7`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:56:37.585099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6d0a0243-b7b5-4c9b-ae00-b22d58a79910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:37.652313 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1049/2082] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:56:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:56:37.656817 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67ed00-4150-45bc-9d34-d4ac0743e872 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:37.720797 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67ed00-4150-45bc-9d34-d4ac0743e872 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:37.726099 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359220a-9611-41e8-ac9c-8431175a6561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:37.726698 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359220a-9611-41e8-ac9c-8431175a6561 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:37.726973 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359220a-9611-41e8-ac9c-8431175a6561 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:37.727241 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359220a-9611-41e8-ac9c-8431175a6561 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:37.727909 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7359220a-9611-41e8-ac9c-8431175a6561 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:37.729802 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7359220a-9611-41e8-ac9c-8431175a6561 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:37.814793 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1034/2083] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:56:37 2022] POST /identity/v3/projects => generated 385 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:56:37.818790 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a0243-b7b5-4c9b-ae00-b22d58a79910 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:37.824250 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:37.829949 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a0243-b7b5-4c9b-ae00-b22d58a79910 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:37.885810 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec3193-d717-4c1a-a433-506010d6d459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:37.886442 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec3193-d717-4c1a-a433-506010d6d459 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:37.886720 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec3193-d717-4c1a-a433-506010d6d459 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:37.887019 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec3193-d717-4c1a-a433-506010d6d459 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:37.887646 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec3193-d717-4c1a-a433-506010d6d459 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:37.889563 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec3193-d717-4c1a-a433-506010d6d459 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:37.986894 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1050/2084] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:56:37 2022] POST /identity/v3/users => generated 299 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:56:37.990963 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359220a-9611-41e8-ac9c-8431175a6561 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:38.001037 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359220a-9611-41e8-ac9c-8431175a6561 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:38.005847 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f4763-8492-4025-9b21-c3a37d471b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:38.006435 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f4763-8492-4025-9b21-c3a37d471b73 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:38.006712 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f4763-8492-4025-9b21-c3a37d471b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:38.006981 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349f4763-8492-4025-9b21-c3a37d471b73 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:38.007621 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f4763-8492-4025-9b21-c3a37d471b73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:38.009213 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349f4763-8492-4025-9b21-c3a37d471b73 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:38.020105 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1035/2085] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:56:38.024141 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec3193-d717-4c1a-a433-506010d6d459 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:38.034677 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec3193-d717-4c1a-a433-506010d6d459 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:38.039527 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0feae-06d8-4914-bd07-c121dd9c17d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:38.040213 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0feae-06d8-4914-bd07-c121dd9c17d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:38.040491 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0feae-06d8-4914-bd07-c121dd9c17d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:38.040765 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0feae-06d8-4914-bd07-c121dd9c17d9 admin admin] PATH_INFO: `/v3/projects/331545ba36914a2f89ab4fe41b908afc/users/0f2301b163574062b25f3cf8d17349d4/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:38.055212 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded0feae-06d8-4914-bd07-c121dd9c17d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=331545ba36914a2f89ab4fe41b908afc, user_id=0f2301b163574062b25f3cf8d17349d4, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:38.057875 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded0feae-06d8-4914-bd07-c121dd9c17d9 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:38.147776 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1051/2086] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:56:38 2022] PUT /identity/v3/projects/331545ba36914a2f89ab4fe41b908afc/users/0f2301b163574062b25f3cf8d17349d4/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:56:38.151651 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f4763-8492-4025-9b21-c3a37d471b73 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:38.162094 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349f4763-8492-4025-9b21-c3a37d471b73 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:38.189490 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:38.217092 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b053b4-948b-41e2-9238-77957b6f3fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:38.217692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b053b4-948b-41e2-9238-77957b6f3fcb admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:38.217971 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b053b4-948b-41e2-9238-77957b6f3fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:38.218239 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b053b4-948b-41e2-9238-77957b6f3fcb admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:38.218867 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b053b4-948b-41e2-9238-77957b6f3fcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:38.220528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b053b4-948b-41e2-9238-77957b6f3fcb admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:38.231112 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1036/2087] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:56:38.235289 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0feae-06d8-4914-bd07-c121dd9c17d9 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:38.245634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0feae-06d8-4914-bd07-c121dd9c17d9 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:38.250294 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2a2dd-0563-4c58-869e-f982f084b3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:38.250965 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2a2dd-0563-4c58-869e-f982f084b3aa admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:38.251248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2a2dd-0563-4c58-869e-f982f084b3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:38.251522 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2a2dd-0563-4c58-869e-f982f084b3aa admin admin] PATH_INFO: `/v3/projects/331545ba36914a2f89ab4fe41b908afc/users/0f2301b163574062b25f3cf8d17349d4/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:38.253523 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2a2dd-0563-4c58-869e-f982f084b3aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=331545ba36914a2f89ab4fe41b908afc, user_id=0f2301b163574062b25f3cf8d17349d4, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:38.256262 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2a2dd-0563-4c58-869e-f982f084b3aa admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:38.266529 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1052/2088] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:56:38 2022] PUT /identity/v3/projects/331545ba36914a2f89ab4fe41b908afc/users/0f2301b163574062b25f3cf8d17349d4/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:56:38.277914 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2df7b-dc4b-4dfe-95d5-488dbecc6b1b None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:38.278200 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2df7b-dc4b-4dfe-95d5-488dbecc6b1b None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:38.278473 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2df7b-dc4b-4dfe-95d5-488dbecc6b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:38.306095 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-cbd2df7b-dc4b-4dfe-95d5-488dbecc6b1b None None] MFA Rules not processed for user `0f2301b163574062b25f3cf8d17349d4`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:56:38.307281 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-cbd2df7b-dc4b-4dfe-95d5-488dbecc6b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:38.372855 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1037/2089] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:56:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:56:38.384486 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2a2dd-0563-4c58-869e-f982f084b3aa admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:38.386173 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-72a2a2dd-0563-4c58-869e-f982f084b3aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:38.442362 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2a2dd-0563-4c58-869e-f982f084b3aa admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:38.445841 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4259a8e-230c-4d39-8a07-d0bf43c9c7ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:38.446458 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4259a8e-230c-4d39-8a07-d0bf43c9c7ff service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:38.446735 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4259a8e-230c-4d39-8a07-d0bf43c9c7ff service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:38.447002 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4259a8e-230c-4d39-8a07-d0bf43c9c7ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:38.501460 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4259a8e-230c-4d39-8a07-d0bf43c9c7ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:38.503382 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4259a8e-230c-4d39-8a07-d0bf43c9c7ff service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:38.512065 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1053/2090] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:56:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:56:40.055247 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94281f2a-acf1-479b-8cf9-3d9fa6ca2639 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:40.055772 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94281f2a-acf1-479b-8cf9-3d9fa6ca2639 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:40.055917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94281f2a-acf1-479b-8cf9-3d9fa6ca2639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:40.076846 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-94281f2a-acf1-479b-8cf9-3d9fa6ca2639 None None] MFA Rules not processed for user `cecbc19355ef4570b2b05bba876a3a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:56:40.078239 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-94281f2a-acf1-479b-8cf9-3d9fa6ca2639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:40.137375 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1038/2091] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Oct 17 03:56:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:56:40.314633 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4259a8e-230c-4d39-8a07-d0bf43c9c7ff service cinder] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:40.316688 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-b4259a8e-230c-4d39-8a07-d0bf43c9c7ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:40.327937 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:40.388547 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4259a8e-230c-4d39-8a07-d0bf43c9c7ff service cinder] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:40.393599 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5bcbca-2453-42e2-8078-17ac87a0019b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f64aaf4d408462a88963973d42abfb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:40.395609 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bcbca-2453-42e2-8078-17ac87a0019b service swift] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:40.395609 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bcbca-2453-42e2-8078-17ac87a0019b service swift] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:40.395609 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5bcbca-2453-42e2-8078-17ac87a0019b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:40.403379 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5bcbca-2453-42e2-8078-17ac87a0019b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:40.405008 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5bcbca-2453-42e2-8078-17ac87a0019b service swift] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:40.413173 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1054/2092] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Oct 17 03:56:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 03:56:41.730072 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94281f2a-acf1-479b-8cf9-3d9fa6ca2639 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:41.731716 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-94281f2a-acf1-479b-8cf9-3d9fa6ca2639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:41.796931 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:41.824782 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94281f2a-acf1-479b-8cf9-3d9fa6ca2639 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:41.838699 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a50081-92f8-4c8d-bd40-e2399edbc180 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:41.839563 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a50081-92f8-4c8d-bd40-e2399edbc180 service nova] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:41.840006 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a50081-92f8-4c8d-bd40-e2399edbc180 service nova] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:41.840373 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a50081-92f8-4c8d-bd40-e2399edbc180 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:41.842960 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-44a50081-92f8-4c8d-bd40-e2399edbc180 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:41.865167 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a50081-92f8-4c8d-bd40-e2399edbc180 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:41.866943 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a50081-92f8-4c8d-bd40-e2399edbc180 service nova] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:41.882231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1039/2093] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:56:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:56:43.872585 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5bcbca-2453-42e2-8078-17ac87a0019b service swift] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:43.873997 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-1e5bcbca-2453-42e2-8078-17ac87a0019b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:43.886585 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5bcbca-2453-42e2-8078-17ac87a0019b service swift] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:43.890254 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b070e8-ce67-4755-a556-c272bafb565e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:43.890856 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b070e8-ce67-4755-a556-c272bafb565e service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:43.891130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b070e8-ce67-4755-a556-c272bafb565e service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:43.891393 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b070e8-ce67-4755-a556-c272bafb565e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:43.902901 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b070e8-ce67-4755-a556-c272bafb565e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:43.904514 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b070e8-ce67-4755-a556-c272bafb565e service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:43.915232 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1055/2094] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:56:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:56:50.373389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a50081-92f8-4c8d-bd40-e2399edbc180 service nova] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:50.444076 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a50081-92f8-4c8d-bd40-e2399edbc180 service nova] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:50.448622 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238227c-e5f0-4d30-9191-51fa6bdfb4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:50.449241 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238227c-e5f0-4d30-9191-51fa6bdfb4d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:50.449518 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238227c-e5f0-4d30-9191-51fa6bdfb4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:50.449819 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d238227c-e5f0-4d30-9191-51fa6bdfb4d3 admin admin] PATH_INFO: `/v3/users/f2310da421c64dada405f3a09b550bd7` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:50.450923 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d238227c-e5f0-4d30-9191-51fa6bdfb4d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2310da421c64dada405f3a09b550bd7)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:50.452881 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d238227c-e5f0-4d30-9191-51fa6bdfb4d3 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:50.655632 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-d238227c-e5f0-4d30-9191-51fa6bdfb4d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2310da421c64dada405f3a09b550bd7'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:50.761870 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-d238227c-e5f0-4d30-9191-51fa6bdfb4d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2310da421c64dada405f3a09b550bd7'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:50.776824 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-d238227c-e5f0-4d30-9191-51fa6bdfb4d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2310da421c64dada405f3a09b550bd7'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:50.777522 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-d238227c-e5f0-4d30-9191-51fa6bdfb4d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2310da421c64dada405f3a09b550bd7'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:50.778159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-d238227c-e5f0-4d30-9191-51fa6bdfb4d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2310da421c64dada405f3a09b550bd7'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:50.798905 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1040/2095] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:56:50 2022] DELETE /identity/v3/users/f2310da421c64dada405f3a09b550bd7 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:56:50.803043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156c2e3-e4de-4632-b862-61402ae9dadb None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:50.803505 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156c2e3-e4de-4632-b862-61402ae9dadb None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:50.803786 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156c2e3-e4de-4632-b862-61402ae9dadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:50.823289 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-7156c2e3-e4de-4632-b862-61402ae9dadb None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:56:50.824525 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-7156c2e3-e4de-4632-b862-61402ae9dadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:50.883559 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1056/2096] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:56:50.893337 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238227c-e5f0-4d30-9191-51fa6bdfb4d3 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:50.896021 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-d238227c-e5f0-4d30-9191-51fa6bdfb4d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:50.957785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d238227c-e5f0-4d30-9191-51fa6bdfb4d3 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:50.963406 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dafac3-e7fc-4853-ac82-68cf303fe819 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:50.964103 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dafac3-e7fc-4853-ac82-68cf303fe819 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:50.964363 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dafac3-e7fc-4853-ac82-68cf303fe819 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:50.964633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dafac3-e7fc-4853-ac82-68cf303fe819 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:50.974766 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dafac3-e7fc-4853-ac82-68cf303fe819 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:50.976249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dafac3-e7fc-4853-ac82-68cf303fe819 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:50.987023 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1041/2097] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:56:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:56:51.461444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156c2e3-e4de-4632-b862-61402ae9dadb None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:51.462902 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-7156c2e3-e4de-4632-b862-61402ae9dadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:51.476923 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156c2e3-e4de-4632-b862-61402ae9dadb None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:51.482723 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf9f0b-a3ab-4595-bf72-7be049d182e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:51.491067 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf9f0b-a3ab-4595-bf72-7be049d182e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:51.491380 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf9f0b-a3ab-4595-bf72-7be049d182e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:51.491664 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf9f0b-a3ab-4595-bf72-7be049d182e4 admin admin] PATH_INFO: `/v3/projects/ffa18779c9f1446f9f82cb3f6abac921` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:51.493019 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf9f0b-a3ab-4595-bf72-7be049d182e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffa18779c9f1446f9f82cb3f6abac921)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:51.495906 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf9f0b-a3ab-4595-bf72-7be049d182e4 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:51.620047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ecaf9f0b-a3ab-4595-bf72-7be049d182e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffa18779c9f1446f9f82cb3f6abac921', 'request_id': 'req-ecaf9f0b-a3ab-4595-bf72-7be049d182e4'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:51.696328 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ecaf9f0b-a3ab-4595-bf72-7be049d182e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffa18779c9f1446f9f82cb3f6abac921', 'request_id': 'req-ecaf9f0b-a3ab-4595-bf72-7be049d182e4'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:51.702719 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ecaf9f0b-a3ab-4595-bf72-7be049d182e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffa18779c9f1446f9f82cb3f6abac921', 'request_id': 'req-ecaf9f0b-a3ab-4595-bf72-7be049d182e4'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:51.714339 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ecaf9f0b-a3ab-4595-bf72-7be049d182e4 admin admin] The token cache is being invalidate because project ffa18779c9f1446f9f82cb3f6abac921 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:56:51.714626 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ecaf9f0b-a3ab-4595-bf72-7be049d182e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:51.715299 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ecaf9f0b-a3ab-4595-bf72-7be049d182e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:51.716883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1057/2098] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:56:51 2022] DELETE /identity/v3/projects/ffa18779c9f1446f9f82cb3f6abac921 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:56:51.721036 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dafac3-e7fc-4853-ac82-68cf303fe819 service neutron] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:51.727074 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:51.731116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dafac3-e7fc-4853-ac82-68cf303fe819 service neutron] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:51.787170 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebb6db-6210-4467-9f85-6d2018fd315e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:51.787819 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebb6db-6210-4467-9f85-6d2018fd315e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:51.788104 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebb6db-6210-4467-9f85-6d2018fd315e admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:51.788371 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebb6db-6210-4467-9f85-6d2018fd315e admin admin] PATH_INFO: `/v3/users/0f2301b163574062b25f3cf8d17349d4` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:51.789511 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ebb6db-6210-4467-9f85-6d2018fd315e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f2301b163574062b25f3cf8d17349d4)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:51.792313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ebb6db-6210-4467-9f85-6d2018fd315e admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:51.946415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-a3ebb6db-6210-4467-9f85-6d2018fd315e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f2301b163574062b25f3cf8d17349d4'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:52.015892 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-a3ebb6db-6210-4467-9f85-6d2018fd315e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f2301b163574062b25f3cf8d17349d4'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:52.027818 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-a3ebb6db-6210-4467-9f85-6d2018fd315e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f2301b163574062b25f3cf8d17349d4'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:52.028432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-a3ebb6db-6210-4467-9f85-6d2018fd315e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f2301b163574062b25f3cf8d17349d4'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:52.029022 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-a3ebb6db-6210-4467-9f85-6d2018fd315e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f2301b163574062b25f3cf8d17349d4'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:52.041490 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1042/2099] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:56:51 2022] DELETE /identity/v3/users/0f2301b163574062b25f3cf8d17349d4 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:56:52.464367 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf9f0b-a3ab-4595-bf72-7be049d182e4 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:52.465809 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-ecaf9f0b-a3ab-4595-bf72-7be049d182e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:52.480637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf9f0b-a3ab-4595-bf72-7be049d182e4 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:52.485653 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:52.536238 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ddbbc1-f054-4042-b8cb-c3aabe09e25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:52.536862 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ddbbc1-f054-4042-b8cb-c3aabe09e25a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:52.537140 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ddbbc1-f054-4042-b8cb-c3aabe09e25a admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:52.537408 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ddbbc1-f054-4042-b8cb-c3aabe09e25a admin admin] PATH_INFO: `/v3/projects/331545ba36914a2f89ab4fe41b908afc` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:52.538509 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ddbbc1-f054-4042-b8cb-c3aabe09e25a admin admin] RBAC: Authorizing `identity:delete_project(project_id=331545ba36914a2f89ab4fe41b908afc)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:52.540438 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ddbbc1-f054-4042-b8cb-c3aabe09e25a admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:52.639839 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-e7ddbbc1-f054-4042-b8cb-c3aabe09e25a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '331545ba36914a2f89ab4fe41b908afc', 'request_id': 'req-e7ddbbc1-f054-4042-b8cb-c3aabe09e25a'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:52.702107 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-e7ddbbc1-f054-4042-b8cb-c3aabe09e25a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '331545ba36914a2f89ab4fe41b908afc', 'request_id': 'req-e7ddbbc1-f054-4042-b8cb-c3aabe09e25a'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:52.709861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-e7ddbbc1-f054-4042-b8cb-c3aabe09e25a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '331545ba36914a2f89ab4fe41b908afc', 'request_id': 'req-e7ddbbc1-f054-4042-b8cb-c3aabe09e25a'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:52.721823 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-e7ddbbc1-f054-4042-b8cb-c3aabe09e25a admin admin] The token cache is being invalidate because project 331545ba36914a2f89ab4fe41b908afc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:56:52.722117 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-e7ddbbc1-f054-4042-b8cb-c3aabe09e25a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:52.722774 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-e7ddbbc1-f054-4042-b8cb-c3aabe09e25a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:52.724289 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1058/2100] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:56:52 2022] DELETE /identity/v3/projects/331545ba36914a2f89ab4fe41b908afc => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:56:52.730195 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bf70b-a488-4b10-aa63-99afcb7d707c None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:52.730650 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bf70b-a488-4b10-aa63-99afcb7d707c None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:52.730761 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bf70b-a488-4b10-aa63-99afcb7d707c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:52.750849 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-561bf70b-a488-4b10-aa63-99afcb7d707c None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:56:52.752075 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-561bf70b-a488-4b10-aa63-99afcb7d707c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:52.809526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1043/2101] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:56:52.818515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46edb2-4870-432a-b069-b9a16cdaf1fa None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:52.818883 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46edb2-4870-432a-b069-b9a16cdaf1fa None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:52.819219 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a46edb2-4870-432a-b069-b9a16cdaf1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:52.839669 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-3a46edb2-4870-432a-b069-b9a16cdaf1fa None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:56:52.840900 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-3a46edb2-4870-432a-b069-b9a16cdaf1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:52.848841 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1059/2102] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:56:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:56:52.853164 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bf70b-a488-4b10-aa63-99afcb7d707c None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:52.862962 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bf70b-a488-4b10-aa63-99afcb7d707c None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:52.867495 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b725ef6-dcf2-4ef4-a02b-5e1cc4322faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:52.868120 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b725ef6-dcf2-4ef4-a02b-5e1cc4322faa admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:52.868402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b725ef6-dcf2-4ef4-a02b-5e1cc4322faa admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:52.868672 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b725ef6-dcf2-4ef4-a02b-5e1cc4322faa admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:52.869317 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b725ef6-dcf2-4ef4-a02b-5e1cc4322faa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:56:52.869616 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b725ef6-dcf2-4ef4-a02b-5e1cc4322faa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:52.871187 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b725ef6-dcf2-4ef4-a02b-5e1cc4322faa admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:52.886427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1044/2103] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:56:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:56:52.887604 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46edb2-4870-432a-b069-b9a16cdaf1fa None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:52.897476 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a46edb2-4870-432a-b069-b9a16cdaf1fa None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:52.902124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7cf0a6-6555-4581-97f4-490fb602c6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:52.902722 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cf0a6-6555-4581-97f4-490fb602c6e2 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:52.903000 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cf0a6-6555-4581-97f4-490fb602c6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:52.903269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7cf0a6-6555-4581-97f4-490fb602c6e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:52.903990 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7cf0a6-6555-4581-97f4-490fb602c6e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:52.905895 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7cf0a6-6555-4581-97f4-490fb602c6e2 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:53.035328 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1060/2104] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:56:52 2022] POST /identity/v3/projects => generated 375 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:56:53.039269 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b725ef6-dcf2-4ef4-a02b-5e1cc4322faa admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:53.049306 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b725ef6-dcf2-4ef4-a02b-5e1cc4322faa admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:53.058129 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:53.107416 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b1ffb-5128-40d1-83b9-ce4f991166c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:53.108102 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b1ffb-5128-40d1-83b9-ce4f991166c4 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:53.108364 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b1ffb-5128-40d1-83b9-ce4f991166c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:53.108634 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b1ffb-5128-40d1-83b9-ce4f991166c4 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:53.109303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07b1ffb-5128-40d1-83b9-ce4f991166c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:53.111229 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07b1ffb-5128-40d1-83b9-ce4f991166c4 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:53.231062 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1045/2105] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:56:53 2022] POST /identity/v3/users => generated 288 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:56:53.239797 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7cf0a6-6555-4581-97f4-490fb602c6e2 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:53.246516 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7cf0a6-6555-4581-97f4-490fb602c6e2 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:53.251168 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cec2d7-1615-46fd-a746-f3592940b1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:53.251851 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cec2d7-1615-46fd-a746-f3592940b1ba admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:53.252108 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cec2d7-1615-46fd-a746-f3592940b1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:53.252379 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cec2d7-1615-46fd-a746-f3592940b1ba admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:53.253016 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cec2d7-1615-46fd-a746-f3592940b1ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:53.255247 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cec2d7-1615-46fd-a746-f3592940b1ba admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:53.266769 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1061/2106] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:56:53.270731 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b1ffb-5128-40d1-83b9-ce4f991166c4 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:53.284017 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b1ffb-5128-40d1-83b9-ce4f991166c4 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:53.286303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3add8-4e14-4711-b51d-c57f17d7d220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:53.286973 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3add8-4e14-4711-b51d-c57f17d7d220 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:53.287319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3add8-4e14-4711-b51d-c57f17d7d220 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:53.287596 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3add8-4e14-4711-b51d-c57f17d7d220 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:53.288279 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce3add8-4e14-4711-b51d-c57f17d7d220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:53.289857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce3add8-4e14-4711-b51d-c57f17d7d220 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:53.301857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1046/2107] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:56:53.305594 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cec2d7-1615-46fd-a746-f3592940b1ba admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:53.315890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cec2d7-1615-46fd-a746-f3592940b1ba admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:53.321448 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc60add-33a1-4849-a42b-bbc40313aaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:53.322181 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc60add-33a1-4849-a42b-bbc40313aaca admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:53.322556 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc60add-33a1-4849-a42b-bbc40313aaca admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:53.322895 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc60add-33a1-4849-a42b-bbc40313aaca admin admin] PATH_INFO: `/v3/projects/27d56a3f43fc402c8dcb2f065aa69baa/users/2e5f0475d4a6486183d864f4b145a696/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:53.338769 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc60add-33a1-4849-a42b-bbc40313aaca admin admin] RBAC: Authorizing `identity:create_grant(project_id=27d56a3f43fc402c8dcb2f065aa69baa, user_id=2e5f0475d4a6486183d864f4b145a696, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:53.341547 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc60add-33a1-4849-a42b-bbc40313aaca admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:53.447427 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1062/2108] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:56:53 2022] PUT /identity/v3/projects/27d56a3f43fc402c8dcb2f065aa69baa/users/2e5f0475d4a6486183d864f4b145a696/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:56:53.455116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03232c1-f47d-4b05-952f-8ddf597ec53d None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:53.461874 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03232c1-f47d-4b05-952f-8ddf597ec53d None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:53.462497 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03232c1-f47d-4b05-952f-8ddf597ec53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:53.500958 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-b03232c1-f47d-4b05-952f-8ddf597ec53d None None] MFA Rules not processed for user `2e5f0475d4a6486183d864f4b145a696`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:56:53.502377 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-b03232c1-f47d-4b05-952f-8ddf597ec53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:53.590915 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1047/2109] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:56:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:56:53.602395 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc60add-33a1-4849-a42b-bbc40313aaca admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:53.604549 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-fdc60add-33a1-4849-a42b-bbc40313aaca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:53.645410 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:53.670265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc60add-33a1-4849-a42b-bbc40313aaca admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:53.674471 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f96339-88ec-427a-82ab-87669b82e667 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:53.675431 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f96339-88ec-427a-82ab-87669b82e667 service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:53.675431 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f96339-88ec-427a-82ab-87669b82e667 service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:53.675660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f96339-88ec-427a-82ab-87669b82e667 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:53.690280 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f96339-88ec-427a-82ab-87669b82e667 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:53.691633 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f96339-88ec-427a-82ab-87669b82e667 service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:53.705102 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1063/2110] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:56:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:56:58.012756 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03232c1-f47d-4b05-952f-8ddf597ec53d None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:58.014295 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-b03232c1-f47d-4b05-952f-8ddf597ec53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:58.037972 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f96339-88ec-427a-82ab-87669b82e667 service cinder] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:58.078402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03232c1-f47d-4b05-952f-8ddf597ec53d None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:58.082102 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc29291-2abd-46b5-9d2f-979be9ceff98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:58.082930 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc29291-2abd-46b5-9d2f-979be9ceff98 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:58.083048 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc29291-2abd-46b5-9d2f-979be9ceff98 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:58.083255 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc29291-2abd-46b5-9d2f-979be9ceff98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:58.085817 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-bfc29291-2abd-46b5-9d2f-979be9ceff98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:58.095889 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc29291-2abd-46b5-9d2f-979be9ceff98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:58.097306 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc29291-2abd-46b5-9d2f-979be9ceff98 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:58.106515 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1048/2111] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:56:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:56:58.106845 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f96339-88ec-427a-82ab-87669b82e667 service cinder] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:58.111588 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108da714-c2c2-498e-988a-8eb226b8eb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:58.112334 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108da714-c2c2-498e-988a-8eb226b8eb99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:58.112688 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108da714-c2c2-498e-988a-8eb226b8eb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:58.113293 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108da714-c2c2-498e-988a-8eb226b8eb99 admin admin] PATH_INFO: `/v3/users/2e5f0475d4a6486183d864f4b145a696` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:58.114460 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108da714-c2c2-498e-988a-8eb226b8eb99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e5f0475d4a6486183d864f4b145a696)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:58.116645 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108da714-c2c2-498e-988a-8eb226b8eb99 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:58.374791 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-108da714-c2c2-498e-988a-8eb226b8eb99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e5f0475d4a6486183d864f4b145a696'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:58.375372 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-108da714-c2c2-498e-988a-8eb226b8eb99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e5f0475d4a6486183d864f4b145a696'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:58.445782 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-108da714-c2c2-498e-988a-8eb226b8eb99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e5f0475d4a6486183d864f4b145a696'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:58.460884 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-108da714-c2c2-498e-988a-8eb226b8eb99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e5f0475d4a6486183d864f4b145a696'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:58.461620 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-108da714-c2c2-498e-988a-8eb226b8eb99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e5f0475d4a6486183d864f4b145a696'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:58.474268 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1064/2112] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:56:58 2022] DELETE /identity/v3/users/2e5f0475d4a6486183d864f4b145a696 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:56:58.478900 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94bd1a-5b7e-47b4-8a5d-f9c7e2e7b97d None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:58.479185 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94bd1a-5b7e-47b4-8a5d-f9c7e2e7b97d None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:58.479455 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94bd1a-5b7e-47b4-8a5d-f9c7e2e7b97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:58.499189 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-cb94bd1a-5b7e-47b4-8a5d-f9c7e2e7b97d None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:56:58.500457 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-cb94bd1a-5b7e-47b4-8a5d-f9c7e2e7b97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:58.518054 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:58.569922 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1049/2113] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:56:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:56:58.591683 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108da714-c2c2-498e-988a-8eb226b8eb99 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:58.593521 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-108da714-c2c2-498e-988a-8eb226b8eb99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:58.653940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108da714-c2c2-498e-988a-8eb226b8eb99 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:58.657545 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9b876-55cd-4402-9f0d-3da6b50a21ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:58.658154 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9b876-55cd-4402-9f0d-3da6b50a21ca service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:58.658436 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9b876-55cd-4402-9f0d-3da6b50a21ca service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:58.658710 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9b876-55cd-4402-9f0d-3da6b50a21ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:58.667800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9b876-55cd-4402-9f0d-3da6b50a21ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:58.669236 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9b876-55cd-4402-9f0d-3da6b50a21ca service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:58.676607 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:56:58.682425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1065/2114] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:56:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:56:59.152256 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb94bd1a-5b7e-47b4-8a5d-f9c7e2e7b97d None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:59.153794 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-cb94bd1a-5b7e-47b4-8a5d-f9c7e2e7b97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:59.166185 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb94bd1a-5b7e-47b4-8a5d-f9c7e2e7b97d None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:59.170798 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4c3be-5276-4a15-be6b-888d2fd6a763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:59.171435 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4c3be-5276-4a15-be6b-888d2fd6a763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:59.171721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4c3be-5276-4a15-be6b-888d2fd6a763 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:59.172039 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4c3be-5276-4a15-be6b-888d2fd6a763 admin admin] PATH_INFO: `/v3/projects/27d56a3f43fc402c8dcb2f065aa69baa` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:59.180483 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4c3be-5276-4a15-be6b-888d2fd6a763 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27d56a3f43fc402c8dcb2f065aa69baa)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:59.183679 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4c3be-5276-4a15-be6b-888d2fd6a763 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:59.338714 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-81f4c3be-5276-4a15-be6b-888d2fd6a763 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27d56a3f43fc402c8dcb2f065aa69baa', 'request_id': 'req-81f4c3be-5276-4a15-be6b-888d2fd6a763'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:59.424795 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-81f4c3be-5276-4a15-be6b-888d2fd6a763 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27d56a3f43fc402c8dcb2f065aa69baa', 'request_id': 'req-81f4c3be-5276-4a15-be6b-888d2fd6a763'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:59.432110 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-81f4c3be-5276-4a15-be6b-888d2fd6a763 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27d56a3f43fc402c8dcb2f065aa69baa', 'request_id': 'req-81f4c3be-5276-4a15-be6b-888d2fd6a763'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:59.443907 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-81f4c3be-5276-4a15-be6b-888d2fd6a763 admin admin] The token cache is being invalidate because project 27d56a3f43fc402c8dcb2f065aa69baa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:56:59.444343 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-81f4c3be-5276-4a15-be6b-888d2fd6a763 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:59.445118 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-81f4c3be-5276-4a15-be6b-888d2fd6a763 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:56:59.447608 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1050/2115] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:56:59 2022] DELETE /identity/v3/projects/27d56a3f43fc402c8dcb2f065aa69baa => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:56:59.453485 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edf31c-5f0e-4db1-a0b1-bef2ed7a0133 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:59.454016 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edf31c-5f0e-4db1-a0b1-bef2ed7a0133 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:59.454016 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6edf31c-5f0e-4db1-a0b1-bef2ed7a0133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:59.475989 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-a6edf31c-5f0e-4db1-a0b1-bef2ed7a0133 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:56:59.477553 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-a6edf31c-5f0e-4db1-a0b1-bef2ed7a0133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:59.545025 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1066/2116] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:56:59.609777 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c0deb-fd1c-4616-8010-09222f3f6c3b None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:59.617347 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c0deb-fd1c-4616-8010-09222f3f6c3b None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:59.617347 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c0deb-fd1c-4616-8010-09222f3f6c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:59.637335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-a99c0deb-fd1c-4616-8010-09222f3f6c3b None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:56:59.638903 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-a99c0deb-fd1c-4616-8010-09222f3f6c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:56:59.649363 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1051/2117] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:56:59.653376 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6edf31c-5f0e-4db1-a0b1-bef2ed7a0133 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:59.665977 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6edf31c-5f0e-4db1-a0b1-bef2ed7a0133 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:59.677798 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a4577-f48c-4ca1-88f8-f62fc581d4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:59.678387 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a4577-f48c-4ca1-88f8-f62fc581d4a3 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:59.678672 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a4577-f48c-4ca1-88f8-f62fc581d4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:59.678943 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a4577-f48c-4ca1-88f8-f62fc581d4a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:59.679602 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a4577-f48c-4ca1-88f8-f62fc581d4a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:56:59.679934 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a4577-f48c-4ca1-88f8-f62fc581d4a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:59.681715 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a4577-f48c-4ca1-88f8-f62fc581d4a3 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:59.695206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1067/2118] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:56:59.699721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c0deb-fd1c-4616-8010-09222f3f6c3b None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:59.710605 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c0deb-fd1c-4616-8010-09222f3f6c3b None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:59.720444 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e33aa5-3b4e-41b8-84b1-76b96fb11c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:59.721081 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e33aa5-3b4e-41b8-84b1-76b96fb11c52 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:59.721362 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e33aa5-3b4e-41b8-84b1-76b96fb11c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:59.721637 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e33aa5-3b4e-41b8-84b1-76b96fb11c52 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:59.722284 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e33aa5-3b4e-41b8-84b1-76b96fb11c52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:59.724251 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e33aa5-3b4e-41b8-84b1-76b96fb11c52 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:59.830871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1052/2119] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:56:59 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:56:59.835727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a4577-f48c-4ca1-88f8-f62fc581d4a3 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:56:59.846187 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a4577-f48c-4ca1-88f8-f62fc581d4a3 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:56:59.911087 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1c288-d784-4d10-9492-c4f46b829f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:56:59.911713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1c288-d784-4d10-9492-c4f46b829f37 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:56:59.912539 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1c288-d784-4d10-9492-c4f46b829f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:56:59.912539 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f1c288-d784-4d10-9492-c4f46b829f37 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:56:59.912965 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f1c288-d784-4d10-9492-c4f46b829f37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:56:59.921694 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f1c288-d784-4d10-9492-c4f46b829f37 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:56:59.996850 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1068/2120] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:56:59 2022] POST /identity/v3/users => generated 287 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:57:00.001161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e33aa5-3b4e-41b8-84b1-76b96fb11c52 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:00.013287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e33aa5-3b4e-41b8-84b1-76b96fb11c52 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:00.017964 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd39d5-7177-4c29-a253-c271e2c170b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:00.018570 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd39d5-7177-4c29-a253-c271e2c170b2 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:00.018850 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd39d5-7177-4c29-a253-c271e2c170b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:00.021881 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dd39d5-7177-4c29-a253-c271e2c170b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:00.021881 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd39d5-7177-4c29-a253-c271e2c170b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:00.021881 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dd39d5-7177-4c29-a253-c271e2c170b2 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:00.035140 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1053/2121] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:57:00.036464 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1c288-d784-4d10-9492-c4f46b829f37 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:00.046885 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f1c288-d784-4d10-9492-c4f46b829f37 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:00.051655 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8cb9e-77bf-42b5-8e01-b33b1f48087a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:00.052291 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8cb9e-77bf-42b5-8e01-b33b1f48087a admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:00.052570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8cb9e-77bf-42b5-8e01-b33b1f48087a admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:00.052844 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8cb9e-77bf-42b5-8e01-b33b1f48087a admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:00.054014 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb8cb9e-77bf-42b5-8e01-b33b1f48087a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:00.055582 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb8cb9e-77bf-42b5-8e01-b33b1f48087a admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:00.071184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1069/2122] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:57:00.071774 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd39d5-7177-4c29-a253-c271e2c170b2 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:00.085917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dd39d5-7177-4c29-a253-c271e2c170b2 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:00.091610 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354cc6e-caa1-4a21-919b-8f2ab9137d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:00.093978 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354cc6e-caa1-4a21-919b-8f2ab9137d31 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:00.093978 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354cc6e-caa1-4a21-919b-8f2ab9137d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:00.093978 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354cc6e-caa1-4a21-919b-8f2ab9137d31 admin admin] PATH_INFO: `/v3/projects/f8c218efa52643fc9ca855e40bf991df/users/0b48f197698c4210b805d332628168f9/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:00.117624 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354cc6e-caa1-4a21-919b-8f2ab9137d31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8c218efa52643fc9ca855e40bf991df, user_id=0b48f197698c4210b805d332628168f9, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:00.120328 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3354cc6e-caa1-4a21-919b-8f2ab9137d31 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:00.215998 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1054/2123] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:57:00 2022] PUT /identity/v3/projects/f8c218efa52643fc9ca855e40bf991df/users/0b48f197698c4210b805d332628168f9/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:57:00.219655 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c83d0-e09e-4107-91a9-858c7d50ca56 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:00.219979 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c83d0-e09e-4107-91a9-858c7d50ca56 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:00.220256 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c83d0-e09e-4107-91a9-858c7d50ca56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:00.249804 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-4e1c83d0-e09e-4107-91a9-858c7d50ca56 None None] MFA Rules not processed for user `0b48f197698c4210b805d332628168f9`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:57:00.251057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-4e1c83d0-e09e-4107-91a9-858c7d50ca56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:00.308474 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:57:00.318173 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1070/2124] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:57:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:57:00.331998 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354cc6e-caa1-4a21-919b-8f2ab9137d31 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:00.333727 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-3354cc6e-caa1-4a21-919b-8f2ab9137d31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:00.385226 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:57:00.397449 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354cc6e-caa1-4a21-919b-8f2ab9137d31 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:00.401442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233c064-be44-402e-9a64-6f91c7cea1f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:00.402053 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233c064-be44-402e-9a64-6f91c7cea1f6 service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:00.402338 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233c064-be44-402e-9a64-6f91c7cea1f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:00.402613 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233c064-be44-402e-9a64-6f91c7cea1f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:00.411352 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f233c064-be44-402e-9a64-6f91c7cea1f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:00.413510 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f233c064-be44-402e-9a64-6f91c7cea1f6 service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:00.422713 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1055/2125] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:57:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:57:00.449115 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c83d0-e09e-4107-91a9-858c7d50ca56 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:00.510846 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c83d0-e09e-4107-91a9-858c7d50ca56 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:00.515824 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e4099-72d2-40a6-b4f3-890e1a1ed52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:00.516461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e4099-72d2-40a6-b4f3-890e1a1ed52e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:00.516737 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e4099-72d2-40a6-b4f3-890e1a1ed52e admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:00.517009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e4099-72d2-40a6-b4f3-890e1a1ed52e admin admin] PATH_INFO: `/v3/users/0b48f197698c4210b805d332628168f9` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:00.518169 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e4099-72d2-40a6-b4f3-890e1a1ed52e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b48f197698c4210b805d332628168f9)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:00.520257 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e4099-72d2-40a6-b4f3-890e1a1ed52e admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:00.727594 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2f8e4099-72d2-40a6-b4f3-890e1a1ed52e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b48f197698c4210b805d332628168f9'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:00.728289 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2f8e4099-72d2-40a6-b4f3-890e1a1ed52e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b48f197698c4210b805d332628168f9'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:00.840018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2f8e4099-72d2-40a6-b4f3-890e1a1ed52e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b48f197698c4210b805d332628168f9'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:00.857411 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2f8e4099-72d2-40a6-b4f3-890e1a1ed52e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b48f197698c4210b805d332628168f9'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:00.859038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2f8e4099-72d2-40a6-b4f3-890e1a1ed52e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b48f197698c4210b805d332628168f9'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:00.872934 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1071/2126] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:57:00 2022] DELETE /identity/v3/users/0b48f197698c4210b805d332628168f9 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:57:00.877923 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113093c5-7772-4bc5-8a7d-f8cc46e9b684 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:00.878210 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113093c5-7772-4bc5-8a7d-f8cc46e9b684 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:00.878682 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113093c5-7772-4bc5-8a7d-f8cc46e9b684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:00.898622 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-113093c5-7772-4bc5-8a7d-f8cc46e9b684 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:57:00.899861 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-113093c5-7772-4bc5-8a7d-f8cc46e9b684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:00.962253 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1056/2127] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:57:00.969482 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e4099-72d2-40a6-b4f3-890e1a1ed52e admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:00.970963 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-2f8e4099-72d2-40a6-b4f3-890e1a1ed52e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:01.035099 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e4099-72d2-40a6-b4f3-890e1a1ed52e admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:01.039833 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c436e89-66f9-4cf1-9530-8f638d3d35c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:01.040448 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c436e89-66f9-4cf1-9530-8f638d3d35c2 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:01.040723 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c436e89-66f9-4cf1-9530-8f638d3d35c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:01.041006 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c436e89-66f9-4cf1-9530-8f638d3d35c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:01.051435 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c436e89-66f9-4cf1-9530-8f638d3d35c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:01.052905 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c436e89-66f9-4cf1-9530-8f638d3d35c2 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:01.063329 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1072/2128] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:57:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:57:01.188460 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0dd5fd-f922-4fce-89aa-38124f201f61 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:01.188749 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0dd5fd-f922-4fce-89aa-38124f201f61 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:01.189027 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0dd5fd-f922-4fce-89aa-38124f201f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:01.212816 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-9a0dd5fd-f922-4fce-89aa-38124f201f61 None None] MFA Rules not processed for user `d1499b45cce347c39855da14d66e9690`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:57:01.214053 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-9a0dd5fd-f922-4fce-89aa-38124f201f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:01.292517 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1057/2129] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:57:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:57:01.650314 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c436e89-66f9-4cf1-9530-8f638d3d35c2 service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:01.651989 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-1c436e89-66f9-4cf1-9530-8f638d3d35c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:01.653379 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5d468-19c6-4930-9b4e-8958245fb301 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:01.653761 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5d468-19c6-4930-9b4e-8958245fb301 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:01.654174 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a5d468-19c6-4930-9b4e-8958245fb301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:01.666492 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c436e89-66f9-4cf1-9530-8f638d3d35c2 service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:01.671429 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc403f76-a91b-4c98-8cf4-6f3715b9533a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:01.673726 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc403f76-a91b-4c98-8cf4-6f3715b9533a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:01.673726 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc403f76-a91b-4c98-8cf4-6f3715b9533a admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:01.673726 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc403f76-a91b-4c98-8cf4-6f3715b9533a admin admin] PATH_INFO: `/v3/projects/f8c218efa52643fc9ca855e40bf991df` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:01.674011 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc403f76-a91b-4c98-8cf4-6f3715b9533a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8c218efa52643fc9ca855e40bf991df)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:01.675727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc403f76-a91b-4c98-8cf4-6f3715b9533a admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:01.677909 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-40a5d468-19c6-4930-9b4e-8958245fb301 None None] MFA Rules not processed for user `cecbc19355ef4570b2b05bba876a3a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:57:01.679264 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-40a5d468-19c6-4930-9b4e-8958245fb301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:01.698792 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:57:01.753590 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1058/2130] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Oct 17 03:57:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:57:01.766694 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5d468-19c6-4930-9b4e-8958245fb301 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:01.768336 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-40a5d468-19c6-4930-9b4e-8958245fb301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:01.811656 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:57:01.840563 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-bc403f76-a91b-4c98-8cf4-6f3715b9533a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8c218efa52643fc9ca855e40bf991df', 'request_id': 'req-bc403f76-a91b-4c98-8cf4-6f3715b9533a'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:01.842989 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a5d468-19c6-4930-9b4e-8958245fb301 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:01.904069 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf1552-6343-4855-99b1-b883047658dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f64aaf4d408462a88963973d42abfb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:01.904695 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf1552-6343-4855-99b1-b883047658dc service swift] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:01.904978 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf1552-6343-4855-99b1-b883047658dc service swift] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:01.905248 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf1552-6343-4855-99b1-b883047658dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:01.924743 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-bc403f76-a91b-4c98-8cf4-6f3715b9533a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8c218efa52643fc9ca855e40bf991df', 'request_id': 'req-bc403f76-a91b-4c98-8cf4-6f3715b9533a'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:01.931406 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-bc403f76-a91b-4c98-8cf4-6f3715b9533a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8c218efa52643fc9ca855e40bf991df', 'request_id': 'req-bc403f76-a91b-4c98-8cf4-6f3715b9533a'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:01.945687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-bc403f76-a91b-4c98-8cf4-6f3715b9533a admin admin] The token cache is being invalidate because project f8c218efa52643fc9ca855e40bf991df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:57:01.946150 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-bc403f76-a91b-4c98-8cf4-6f3715b9533a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:01.952363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-bc403f76-a91b-4c98-8cf4-6f3715b9533a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:01.952363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1073/2131] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:57:01 2022] DELETE /identity/v3/projects/f8c218efa52643fc9ca855e40bf991df => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:57:01.953914 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541fadd-71e9-4a0f-9897-edcdf9ca7e6e None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:01.954181 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541fadd-71e9-4a0f-9897-edcdf9ca7e6e None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:01.954678 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d541fadd-71e9-4a0f-9897-edcdf9ca7e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:01.963906 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf1552-6343-4855-99b1-b883047658dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:01.965495 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf1552-6343-4855-99b1-b883047658dc service swift] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:01.974509 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1059/2132] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Oct 17 03:57:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 17 03:57:01.977205 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-d541fadd-71e9-4a0f-9897-edcdf9ca7e6e None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:57:01.978314 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-d541fadd-71e9-4a0f-9897-edcdf9ca7e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:02.047775 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1074/2133] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:57:02.049933 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00aeac8-02ce-4de8-9482-c4b3167915c3 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:02.050214 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00aeac8-02ce-4de8-9482-c4b3167915c3 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:02.050484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00aeac8-02ce-4de8-9482-c4b3167915c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:02.071160 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-c00aeac8-02ce-4de8-9482-c4b3167915c3 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:57:02.072425 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c00aeac8-02ce-4de8-9482-c4b3167915c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:02.082418 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1060/2134] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:57:02.087172 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541fadd-71e9-4a0f-9897-edcdf9ca7e6e None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:02.099082 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d541fadd-71e9-4a0f-9897-edcdf9ca7e6e None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:02.121042 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd7216-fed3-461c-83ea-c49f3a147099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:02.121670 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd7216-fed3-461c-83ea-c49f3a147099 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:02.121946 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd7216-fed3-461c-83ea-c49f3a147099 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:02.122220 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd7216-fed3-461c-83ea-c49f3a147099 admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:02.122893 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd7216-fed3-461c-83ea-c49f3a147099 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:57:02.123183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd7216-fed3-461c-83ea-c49f3a147099 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:02.124792 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd7216-fed3-461c-83ea-c49f3a147099 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:02.168493 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00aeac8-02ce-4de8-9482-c4b3167915c3 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:02.175016 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1075/2135] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:57:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:57:02.180349 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00aeac8-02ce-4de8-9482-c4b3167915c3 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:02.215442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f21d95-0db0-47f6-b83e-268c9c138e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:02.215442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f21d95-0db0-47f6-b83e-268c9c138e89 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:02.215442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f21d95-0db0-47f6-b83e-268c9c138e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:02.215442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f21d95-0db0-47f6-b83e-268c9c138e89 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:02.217980 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f21d95-0db0-47f6-b83e-268c9c138e89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:02.225153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f21d95-0db0-47f6-b83e-268c9c138e89 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:02.390502 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1061/2136] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:57:02 2022] POST /identity/v3/projects => generated 375 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:57:02.394964 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd7216-fed3-461c-83ea-c49f3a147099 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:02.416813 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd7216-fed3-461c-83ea-c49f3a147099 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:02.514032 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60b624-d1da-4b59-8ae5-0a11bda5cb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:02.514770 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60b624-d1da-4b59-8ae5-0a11bda5cb4a admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:02.515119 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60b624-d1da-4b59-8ae5-0a11bda5cb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:02.515560 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60b624-d1da-4b59-8ae5-0a11bda5cb4a admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:02.516278 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60b624-d1da-4b59-8ae5-0a11bda5cb4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:02.519794 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60b624-d1da-4b59-8ae5-0a11bda5cb4a admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:02.611925 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1076/2137] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:57:02 2022] POST /identity/v3/users => generated 288 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:57:02.616636 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f21d95-0db0-47f6-b83e-268c9c138e89 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:02.627439 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f21d95-0db0-47f6-b83e-268c9c138e89 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:02.632583 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be21fdb-9dee-4d55-94c1-530f4999a1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:02.633202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be21fdb-9dee-4d55-94c1-530f4999a1de admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:02.633488 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be21fdb-9dee-4d55-94c1-530f4999a1de admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:02.633922 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be21fdb-9dee-4d55-94c1-530f4999a1de admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:02.634741 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be21fdb-9dee-4d55-94c1-530f4999a1de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:02.636356 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be21fdb-9dee-4d55-94c1-530f4999a1de admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:02.650263 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1062/2138] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:57:02.651821 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60b624-d1da-4b59-8ae5-0a11bda5cb4a admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:02.664206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60b624-d1da-4b59-8ae5-0a11bda5cb4a admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:02.676596 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58649af2-de68-4d13-bf6a-6e05a88d3c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:02.680762 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58649af2-de68-4d13-bf6a-6e05a88d3c01 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:02.681385 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58649af2-de68-4d13-bf6a-6e05a88d3c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:02.681876 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58649af2-de68-4d13-bf6a-6e05a88d3c01 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:02.682992 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58649af2-de68-4d13-bf6a-6e05a88d3c01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:02.686009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58649af2-de68-4d13-bf6a-6e05a88d3c01 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:02.699567 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1077/2139] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:57:02 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:57:02.704415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be21fdb-9dee-4d55-94c1-530f4999a1de admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:02.716785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be21fdb-9dee-4d55-94c1-530f4999a1de admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:02.729689 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb715f3-e12a-46ad-96e6-20743288d426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:02.730626 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb715f3-e12a-46ad-96e6-20743288d426 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:02.731059 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb715f3-e12a-46ad-96e6-20743288d426 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:02.731339 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb715f3-e12a-46ad-96e6-20743288d426 admin admin] PATH_INFO: `/v3/projects/6be7dccebac241518620e36863b7165d/users/110cc87e9d5b4344a22db2f209755ea7/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:02.762509 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb715f3-e12a-46ad-96e6-20743288d426 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6be7dccebac241518620e36863b7165d, user_id=110cc87e9d5b4344a22db2f209755ea7, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:02.765502 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb715f3-e12a-46ad-96e6-20743288d426 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:02.867863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1063/2140] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:57:02 2022] PUT /identity/v3/projects/6be7dccebac241518620e36863b7165d/users/110cc87e9d5b4344a22db2f209755ea7/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:57:02.872651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b7343-eab6-40f2-a7c1-241e6abdd282 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:02.873059 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b7343-eab6-40f2-a7c1-241e6abdd282 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:02.873551 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635b7343-eab6-40f2-a7c1-241e6abdd282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:02.889485 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:57:02.914616 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-635b7343-eab6-40f2-a7c1-241e6abdd282 None None] MFA Rules not processed for user `110cc87e9d5b4344a22db2f209755ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:57:02.916311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-635b7343-eab6-40f2-a7c1-241e6abdd282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:03.004825 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1078/2141] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:57:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:57:03.015924 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb715f3-e12a-46ad-96e6-20743288d426 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:03.018016 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-deb715f3-e12a-46ad-96e6-20743288d426 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:03.042928 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:57:03.101894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb715f3-e12a-46ad-96e6-20743288d426 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:03.108868 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dbd6ae-adb9-4c7f-9840-143e2b9567c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:03.109497 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbd6ae-adb9-4c7f-9840-143e2b9567c5 service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:03.109773 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbd6ae-adb9-4c7f-9840-143e2b9567c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:03.110066 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbd6ae-adb9-4c7f-9840-143e2b9567c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:03.133180 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dbd6ae-adb9-4c7f-9840-143e2b9567c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:03.134644 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dbd6ae-adb9-4c7f-9840-143e2b9567c5 service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:03.145816 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1064/2142] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:57:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:57:04.940978 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61138152-6172-458b-98ee-60c7f24e1b98 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:04.941504 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61138152-6172-458b-98ee-60c7f24e1b98 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:04.941605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61138152-6172-458b-98ee-60c7f24e1b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:04.964894 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-61138152-6172-458b-98ee-60c7f24e1b98 None None] MFA Rules not processed for user `110cc87e9d5b4344a22db2f209755ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:57:04.966129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-61138152-6172-458b-98ee-60c7f24e1b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:04.973073 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1079/2143] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:57:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:57:13.276534 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=19 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:57:14.958665 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=32 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:57:15.684502 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec992e9-92f6-4ee9-9a64-97d29f6f9027 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:15.686179 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec992e9-92f6-4ee9-9a64-97d29f6f9027 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:15.686477 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec992e9-92f6-4ee9-9a64-97d29f6f9027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:15.713067 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-6ec992e9-92f6-4ee9-9a64-97d29f6f9027 None None] MFA Rules not processed for user `cecbc19355ef4570b2b05bba876a3a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:57:15.714291 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-6ec992e9-92f6-4ee9-9a64-97d29f6f9027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:15.775126 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1065/2144] 10.222.0.17 () {58 vars in 967 bytes} [Mon Oct 17 03:57:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:57:15.788334 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61138152-6172-458b-98ee-60c7f24e1b98 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:15.790123 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-61138152-6172-458b-98ee-60c7f24e1b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:15.862590 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61138152-6172-458b-98ee-60c7f24e1b98 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:15.867609 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2eab9-6da3-4a73-9b09-51140eda2245 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f64aaf4d408462a88963973d42abfb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:15.868426 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2eab9-6da3-4a73-9b09-51140eda2245 service swift] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:15.868713 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2eab9-6da3-4a73-9b09-51140eda2245 service swift] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:15.868989 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2eab9-6da3-4a73-9b09-51140eda2245 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:15.877436 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e2eab9-6da3-4a73-9b09-51140eda2245 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:15.879451 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e2eab9-6da3-4a73-9b09-51140eda2245 service swift] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:15.887989 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1080/2145] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Oct 17 03:57:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 03:57:25.768013 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=25 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:57:25.884643 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=32 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:57:32.399181 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2d0f3-d116-44f8-98ea-a702b46a6735 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:32.399683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2d0f3-d116-44f8-98ea-a702b46a6735 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:32.399815 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2d0f3-d116-44f8-98ea-a702b46a6735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:32.419771 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-bff2d0f3-d116-44f8-98ea-a702b46a6735 None None] MFA Rules not processed for user `cecbc19355ef4570b2b05bba876a3a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:57:32.420963 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-bff2d0f3-d116-44f8-98ea-a702b46a6735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:32.428151 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1066/2146] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Oct 17 03:57:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:57:32.440720 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2eab9-6da3-4a73-9b09-51140eda2245 service swift] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:32.451690 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2eab9-6da3-4a73-9b09-51140eda2245 service swift] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:32.456948 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd251e82-5fee-4bfc-8d7d-38bbeab2caa8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f64aaf4d408462a88963973d42abfb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:32.457556 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd251e82-5fee-4bfc-8d7d-38bbeab2caa8 service swift] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:32.457835 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd251e82-5fee-4bfc-8d7d-38bbeab2caa8 service swift] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:32.458110 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd251e82-5fee-4bfc-8d7d-38bbeab2caa8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:32.466681 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd251e82-5fee-4bfc-8d7d-38bbeab2caa8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:32.468342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd251e82-5fee-4bfc-8d7d-38bbeab2caa8 service swift] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:32.476899 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1081/2147] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Oct 17 03:57:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 03:57:33.384086 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6caad62-4683-49ca-8e21-f2cd446705b4 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:33.384861 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6caad62-4683-49ca-8e21-f2cd446705b4 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:33.385136 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6caad62-4683-49ca-8e21-f2cd446705b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:33.404614 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-c6caad62-4683-49ca-8e21-f2cd446705b4 None None] MFA Rules not processed for user `cecbc19355ef4570b2b05bba876a3a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:57:33.405790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c6caad62-4683-49ca-8e21-f2cd446705b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:33.412648 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1067/2148] 10.222.0.17 () {58 vars in 967 bytes} [Mon Oct 17 03:57:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:57:33.424573 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd251e82-5fee-4bfc-8d7d-38bbeab2caa8 service swift] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:33.435065 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd251e82-5fee-4bfc-8d7d-38bbeab2caa8 service swift] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:33.440018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4583e8-b164-443c-9eb6-d315c08193e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f64aaf4d408462a88963973d42abfb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:33.440718 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4583e8-b164-443c-9eb6-d315c08193e1 service swift] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:33.440881 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4583e8-b164-443c-9eb6-d315c08193e1 service swift] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:33.441394 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4583e8-b164-443c-9eb6-d315c08193e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:33.449612 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4583e8-b164-443c-9eb6-d315c08193e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:33.451168 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4583e8-b164-443c-9eb6-d315c08193e1 service swift] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:33.459462 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1082/2149] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Oct 17 03:57:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 03:57:34.045887 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23514f-7a33-4a79-890a-127a7dce853a None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:34.046180 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23514f-7a33-4a79-890a-127a7dce853a None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:34.046494 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23514f-7a33-4a79-890a-127a7dce853a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:34.065805 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-7a23514f-7a33-4a79-890a-127a7dce853a None None] MFA Rules not processed for user `cecbc19355ef4570b2b05bba876a3a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:57:34.067000 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-7a23514f-7a33-4a79-890a-127a7dce853a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:34.073973 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1068/2150] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Oct 17 03:57:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:57:34.085566 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4583e8-b164-443c-9eb6-d315c08193e1 service swift] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:34.096158 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4583e8-b164-443c-9eb6-d315c08193e1 service swift] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:34.101134 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a24b2f-524a-4f80-845c-51fb85beaf7f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f64aaf4d408462a88963973d42abfb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:34.101729 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a24b2f-524a-4f80-845c-51fb85beaf7f service swift] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:34.102011 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a24b2f-524a-4f80-845c-51fb85beaf7f service swift] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:34.102283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a24b2f-524a-4f80-845c-51fb85beaf7f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:34.110717 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a24b2f-524a-4f80-845c-51fb85beaf7f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:34.112310 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a24b2f-524a-4f80-845c-51fb85beaf7f service swift] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:34.120774 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1083/2151] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Oct 17 03:57:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 03:57:34.674412 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936b3b8-a757-48c6-9087-4219920f3527 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:34.674915 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936b3b8-a757-48c6-9087-4219920f3527 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:34.675011 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936b3b8-a757-48c6-9087-4219920f3527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:34.694333 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-1936b3b8-a757-48c6-9087-4219920f3527 None None] MFA Rules not processed for user `cecbc19355ef4570b2b05bba876a3a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:57:34.695535 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-1936b3b8-a757-48c6-9087-4219920f3527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:34.703636 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1069/2152] 10.222.0.17 () {58 vars in 967 bytes} [Mon Oct 17 03:57:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:57:34.717932 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a24b2f-524a-4f80-845c-51fb85beaf7f service swift] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:34.728511 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a24b2f-524a-4f80-845c-51fb85beaf7f service swift] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:34.733913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af734dc-03bd-4862-8e7b-839836e62623 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f64aaf4d408462a88963973d42abfb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:34.734503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af734dc-03bd-4862-8e7b-839836e62623 service swift] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:34.734779 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af734dc-03bd-4862-8e7b-839836e62623 service swift] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:34.735049 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af734dc-03bd-4862-8e7b-839836e62623 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:34.743110 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af734dc-03bd-4862-8e7b-839836e62623 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:34.745325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af734dc-03bd-4862-8e7b-839836e62623 service swift] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:34.754071 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1084/2153] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Oct 17 03:57:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 03:57:37.496187 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936b3b8-a757-48c6-9087-4219920f3527 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:37.557522 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936b3b8-a757-48c6-9087-4219920f3527 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:37.562592 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64736af5-391f-4a50-ac6d-758445c63ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:37.563214 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64736af5-391f-4a50-ac6d-758445c63ef2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:37.563494 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64736af5-391f-4a50-ac6d-758445c63ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:37.563789 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64736af5-391f-4a50-ac6d-758445c63ef2 admin admin] PATH_INFO: `/v3/users/110cc87e9d5b4344a22db2f209755ea7` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:37.564906 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64736af5-391f-4a50-ac6d-758445c63ef2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=110cc87e9d5b4344a22db2f209755ea7)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:37.566838 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64736af5-391f-4a50-ac6d-758445c63ef2 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:37.757741 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-64736af5-391f-4a50-ac6d-758445c63ef2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '110cc87e9d5b4344a22db2f209755ea7'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:37.816647 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-64736af5-391f-4a50-ac6d-758445c63ef2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '110cc87e9d5b4344a22db2f209755ea7'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:37.829359 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-64736af5-391f-4a50-ac6d-758445c63ef2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '110cc87e9d5b4344a22db2f209755ea7'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:37.830023 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-64736af5-391f-4a50-ac6d-758445c63ef2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '110cc87e9d5b4344a22db2f209755ea7'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:37.830625 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-64736af5-391f-4a50-ac6d-758445c63ef2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '110cc87e9d5b4344a22db2f209755ea7'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:37.843053 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1070/2154] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:57:37 2022] DELETE /identity/v3/users/110cc87e9d5b4344a22db2f209755ea7 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:57:37.847516 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40effe57-47c2-4354-97ec-d6c50f430a7e None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:37.848082 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40effe57-47c2-4354-97ec-d6c50f430a7e None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:37.848082 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40effe57-47c2-4354-97ec-d6c50f430a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:37.870182 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-40effe57-47c2-4354-97ec-d6c50f430a7e None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:57:37.872185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-40effe57-47c2-4354-97ec-d6c50f430a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:37.930339 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1085/2155] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:57:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:57:37.940620 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64736af5-391f-4a50-ac6d-758445c63ef2 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:37.942318 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-64736af5-391f-4a50-ac6d-758445c63ef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:38.002033 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64736af5-391f-4a50-ac6d-758445c63ef2 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:38.005712 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51ca3e-3fb5-4269-8461-fdb4df66a414 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:38.006320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51ca3e-3fb5-4269-8461-fdb4df66a414 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:38.006603 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51ca3e-3fb5-4269-8461-fdb4df66a414 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:38.006926 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51ca3e-3fb5-4269-8461-fdb4df66a414 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:38.014115 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:57:38.016678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e51ca3e-3fb5-4269-8461-fdb4df66a414 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:38.018095 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e51ca3e-3fb5-4269-8461-fdb4df66a414 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:38.027822 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1071/2156] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:57:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:57:38.369095 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40effe57-47c2-4354-97ec-d6c50f430a7e None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:38.370533 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-40effe57-47c2-4354-97ec-d6c50f430a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:38.384616 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40effe57-47c2-4354-97ec-d6c50f430a7e None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:38.396970 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac162fba-fb02-43a6-8b86-27542525443a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:38.397995 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac162fba-fb02-43a6-8b86-27542525443a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:38.398289 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac162fba-fb02-43a6-8b86-27542525443a admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:38.398570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac162fba-fb02-43a6-8b86-27542525443a admin admin] PATH_INFO: `/v3/projects/6be7dccebac241518620e36863b7165d` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:38.400000 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac162fba-fb02-43a6-8b86-27542525443a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6be7dccebac241518620e36863b7165d)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:38.402604 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac162fba-fb02-43a6-8b86-27542525443a admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:38.517175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ac162fba-fb02-43a6-8b86-27542525443a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6be7dccebac241518620e36863b7165d', 'request_id': 'req-ac162fba-fb02-43a6-8b86-27542525443a'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:38.596171 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ac162fba-fb02-43a6-8b86-27542525443a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6be7dccebac241518620e36863b7165d', 'request_id': 'req-ac162fba-fb02-43a6-8b86-27542525443a'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:38.602073 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ac162fba-fb02-43a6-8b86-27542525443a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6be7dccebac241518620e36863b7165d', 'request_id': 'req-ac162fba-fb02-43a6-8b86-27542525443a'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:38.613441 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ac162fba-fb02-43a6-8b86-27542525443a admin admin] The token cache is being invalidate because project 6be7dccebac241518620e36863b7165d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:57:38.613662 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ac162fba-fb02-43a6-8b86-27542525443a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:38.614285 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ac162fba-fb02-43a6-8b86-27542525443a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:38.615832 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1086/2157] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:57:38 2022] DELETE /identity/v3/projects/6be7dccebac241518620e36863b7165d => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:57:42.200880 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51ca3e-3fb5-4269-8461-fdb4df66a414 service neutron] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:42.209373 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51ca3e-3fb5-4269-8461-fdb4df66a414 service neutron] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:42.258378 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e9705-7fea-4952-b092-3281b7007acf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:42.258980 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e9705-7fea-4952-b092-3281b7007acf service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:42.259259 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e9705-7fea-4952-b092-3281b7007acf service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:42.259530 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e9705-7fea-4952-b092-3281b7007acf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:42.261268 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-b02e9705-7fea-4952-b092-3281b7007acf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:42.321638 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02e9705-7fea-4952-b092-3281b7007acf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:42.323042 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02e9705-7fea-4952-b092-3281b7007acf service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:42.333012 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1072/2158] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:57:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:57:45.960609 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac162fba-fb02-43a6-8b86-27542525443a admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:45.962349 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-ac162fba-fb02-43a6-8b86-27542525443a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:45.977239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac162fba-fb02-43a6-8b86-27542525443a admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:45.981992 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cfe35a-437a-4e07-8739-e0d0cdc89c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:45.982625 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfe35a-437a-4e07-8739-e0d0cdc89c98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:45.982906 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfe35a-437a-4e07-8739-e0d0cdc89c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:45.983191 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cfe35a-437a-4e07-8739-e0d0cdc89c98 admin admin] PATH_INFO: `/v3/users/6f1245ffc45b4c99990e746068bd852c` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:45.984336 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cfe35a-437a-4e07-8739-e0d0cdc89c98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f1245ffc45b4c99990e746068bd852c)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:45.986279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cfe35a-437a-4e07-8739-e0d0cdc89c98 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:46.111274 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:57:46.227191 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-65cfe35a-437a-4e07-8739-e0d0cdc89c98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f1245ffc45b4c99990e746068bd852c'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:46.227870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-65cfe35a-437a-4e07-8739-e0d0cdc89c98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f1245ffc45b4c99990e746068bd852c'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:46.304784 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-65cfe35a-437a-4e07-8739-e0d0cdc89c98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f1245ffc45b4c99990e746068bd852c'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:46.317367 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-65cfe35a-437a-4e07-8739-e0d0cdc89c98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f1245ffc45b4c99990e746068bd852c'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:46.318000 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-65cfe35a-437a-4e07-8739-e0d0cdc89c98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f1245ffc45b4c99990e746068bd852c'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:46.330909 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1087/2159] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:57:45 2022] DELETE /identity/v3/users/6f1245ffc45b4c99990e746068bd852c => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:57:46.600300 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e9705-7fea-4952-b092-3281b7007acf service neutron] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:46.608355 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e9705-7fea-4952-b092-3281b7007acf service neutron] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:46.667129 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdabf8c-9da2-4e2e-864c-cf06167b0b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:46.667790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdabf8c-9da2-4e2e-864c-cf06167b0b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:46.668122 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdabf8c-9da2-4e2e-864c-cf06167b0b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:46.668398 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdabf8c-9da2-4e2e-864c-cf06167b0b3b admin admin] PATH_INFO: `/v3/projects/735cb7f647604741982528fc91abb7b6` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:46.670065 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdabf8c-9da2-4e2e-864c-cf06167b0b3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=735cb7f647604741982528fc91abb7b6)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:46.672007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdabf8c-9da2-4e2e-864c-cf06167b0b3b admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:46.783904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:57:46.785218 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-4cdabf8c-9da2-4e2e-864c-cf06167b0b3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '735cb7f647604741982528fc91abb7b6', 'request_id': 'req-4cdabf8c-9da2-4e2e-864c-cf06167b0b3b'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:46.862623 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-4cdabf8c-9da2-4e2e-864c-cf06167b0b3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '735cb7f647604741982528fc91abb7b6', 'request_id': 'req-4cdabf8c-9da2-4e2e-864c-cf06167b0b3b'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:46.868443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-4cdabf8c-9da2-4e2e-864c-cf06167b0b3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '735cb7f647604741982528fc91abb7b6', 'request_id': 'req-4cdabf8c-9da2-4e2e-864c-cf06167b0b3b'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:46.880309 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-4cdabf8c-9da2-4e2e-864c-cf06167b0b3b admin admin] The token cache is being invalidate because project 735cb7f647604741982528fc91abb7b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:57:46.880621 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-4cdabf8c-9da2-4e2e-864c-cf06167b0b3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:46.881366 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-4cdabf8c-9da2-4e2e-864c-cf06167b0b3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:46.882836 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1073/2160] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:57:46 2022] DELETE /identity/v3/projects/735cb7f647604741982528fc91abb7b6 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:57:46.887803 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a437d-1620-40eb-8cab-1d9a623623b4 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:46.898255 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a437d-1620-40eb-8cab-1d9a623623b4 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:46.898255 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a437d-1620-40eb-8cab-1d9a623623b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:46.907957 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-a76a437d-1620-40eb-8cab-1d9a623623b4 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:57:46.909168 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-a76a437d-1620-40eb-8cab-1d9a623623b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:46.969644 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1088/2161] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:57:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:57:46.977718 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2a5b0-ac82-4872-b812-8c0c2d897020 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:46.978008 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2a5b0-ac82-4872-b812-8c0c2d897020 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:46.978283 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2a5b0-ac82-4872-b812-8c0c2d897020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:46.999394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-7fd2a5b0-ac82-4872-b812-8c0c2d897020 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:57:47.000620 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-7fd2a5b0-ac82-4872-b812-8c0c2d897020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:47.008404 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1074/2162] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:57:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:57:47.012577 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a437d-1620-40eb-8cab-1d9a623623b4 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:47.022465 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a437d-1620-40eb-8cab-1d9a623623b4 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:47.026983 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ceb2aa-e286-4bbc-b1ce-7630ed4c0602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:47.027582 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ceb2aa-e286-4bbc-b1ce-7630ed4c0602 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:47.027897 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ceb2aa-e286-4bbc-b1ce-7630ed4c0602 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:47.031629 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ceb2aa-e286-4bbc-b1ce-7630ed4c0602 admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:47.031629 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ceb2aa-e286-4bbc-b1ce-7630ed4c0602 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:57:47.031629 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ceb2aa-e286-4bbc-b1ce-7630ed4c0602 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:47.031629 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ceb2aa-e286-4bbc-b1ce-7630ed4c0602 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:47.042788 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1089/2163] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:57:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:57:47.047498 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2a5b0-ac82-4872-b812-8c0c2d897020 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:47.056603 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2a5b0-ac82-4872-b812-8c0c2d897020 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:47.061545 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634765f-58f3-4811-9309-53489d81c1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:47.062137 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634765f-58f3-4811-9309-53489d81c1df admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:47.062418 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634765f-58f3-4811-9309-53489d81c1df admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:47.062693 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b634765f-58f3-4811-9309-53489d81c1df admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:47.063322 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b634765f-58f3-4811-9309-53489d81c1df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:47.065252 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b634765f-58f3-4811-9309-53489d81c1df admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:47.143366 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1075/2164] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:57:47 2022] POST /identity/v3/projects => generated 375 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:57:47.147924 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ceb2aa-e286-4bbc-b1ce-7630ed4c0602 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:47.158503 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ceb2aa-e286-4bbc-b1ce-7630ed4c0602 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:47.214533 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547b50a-f264-4f56-a1fd-945b945582b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:47.215137 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547b50a-f264-4f56-a1fd-945b945582b3 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:47.215418 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547b50a-f264-4f56-a1fd-945b945582b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:47.215696 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3547b50a-f264-4f56-a1fd-945b945582b3 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:47.216415 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3547b50a-f264-4f56-a1fd-945b945582b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:47.218868 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3547b50a-f264-4f56-a1fd-945b945582b3 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:47.299789 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1090/2165] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:57:47 2022] POST /identity/v3/users => generated 288 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:57:47.304057 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634765f-58f3-4811-9309-53489d81c1df admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:47.314391 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b634765f-58f3-4811-9309-53489d81c1df admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:47.318928 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc4dd7-fe33-43bd-aa02-3411279dd95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:47.319514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc4dd7-fe33-43bd-aa02-3411279dd95e admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:47.319819 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc4dd7-fe33-43bd-aa02-3411279dd95e admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:47.320092 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc4dd7-fe33-43bd-aa02-3411279dd95e admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:47.320805 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc4dd7-fe33-43bd-aa02-3411279dd95e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:47.322377 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc4dd7-fe33-43bd-aa02-3411279dd95e admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:47.332260 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1076/2166] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:57:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:57:47.336432 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547b50a-f264-4f56-a1fd-945b945582b3 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:47.346689 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3547b50a-f264-4f56-a1fd-945b945582b3 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:47.351231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce9842-7ca1-48e7-a064-99b7cbc710c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:47.351844 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce9842-7ca1-48e7-a064-99b7cbc710c4 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:47.352121 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce9842-7ca1-48e7-a064-99b7cbc710c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:47.352388 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce9842-7ca1-48e7-a064-99b7cbc710c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:47.353010 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce9842-7ca1-48e7-a064-99b7cbc710c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:47.354675 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce9842-7ca1-48e7-a064-99b7cbc710c4 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:47.365651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1091/2167] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:57:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:57:47.369632 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc4dd7-fe33-43bd-aa02-3411279dd95e admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:47.379470 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc4dd7-fe33-43bd-aa02-3411279dd95e admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:47.384106 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0c161-aa2f-4ea9-94d8-ae128b9a39b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:47.384739 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0c161-aa2f-4ea9-94d8-ae128b9a39b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:47.385015 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0c161-aa2f-4ea9-94d8-ae128b9a39b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:47.385289 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a0c161-aa2f-4ea9-94d8-ae128b9a39b7 admin admin] PATH_INFO: `/v3/projects/2a34bf95f3044db19425033a38dbf5e2/users/d9b8cb43c41743a9a9e3aa1b28f78b61/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:47.399856 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a0c161-aa2f-4ea9-94d8-ae128b9a39b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a34bf95f3044db19425033a38dbf5e2, user_id=d9b8cb43c41743a9a9e3aa1b28f78b61, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:47.402426 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a0c161-aa2f-4ea9-94d8-ae128b9a39b7 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:47.496367 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1077/2168] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:57:47 2022] PUT /identity/v3/projects/2a34bf95f3044db19425033a38dbf5e2/users/d9b8cb43c41743a9a9e3aa1b28f78b61/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:57:47.508263 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9df7e-c242-4e00-8491-e3127f1cb34c None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:47.508554 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9df7e-c242-4e00-8491-e3127f1cb34c None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:47.508847 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee9df7e-c242-4e00-8491-e3127f1cb34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:47.528917 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:57:47.537784 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-1ee9df7e-c242-4e00-8491-e3127f1cb34c None None] MFA Rules not processed for user `d9b8cb43c41743a9a9e3aa1b28f78b61`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:57:47.538982 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-1ee9df7e-c242-4e00-8491-e3127f1cb34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:47.602557 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1092/2169] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:57:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:57:47.614314 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0c161-aa2f-4ea9-94d8-ae128b9a39b7 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:47.616246 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-e7a0c161-aa2f-4ea9-94d8-ae128b9a39b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:47.687549 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a0c161-aa2f-4ea9-94d8-ae128b9a39b7 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:47.693119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e52940-24ae-4082-8370-0a96b661a712 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:47.693722 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e52940-24ae-4082-8370-0a96b661a712 service nova] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:47.694000 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e52940-24ae-4082-8370-0a96b661a712 service nova] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:47.694270 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e52940-24ae-4082-8370-0a96b661a712 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:47.702561 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e52940-24ae-4082-8370-0a96b661a712 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:47.704153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e52940-24ae-4082-8370-0a96b661a712 service nova] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:47.713106 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1078/2170] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:57:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:57:47.930587 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9df7e-c242-4e00-8491-e3127f1cb34c None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:47.992193 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee9df7e-c242-4e00-8491-e3127f1cb34c None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:47.996915 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d547e4a-ae81-4e97-9a06-ea8bec54669d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:47.997531 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d547e4a-ae81-4e97-9a06-ea8bec54669d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:47.997808 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d547e4a-ae81-4e97-9a06-ea8bec54669d admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:47.998572 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d547e4a-ae81-4e97-9a06-ea8bec54669d admin admin] PATH_INFO: `/v3/users/d9b8cb43c41743a9a9e3aa1b28f78b61` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:47.999666 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d547e4a-ae81-4e97-9a06-ea8bec54669d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9b8cb43c41743a9a9e3aa1b28f78b61)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:48.001587 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d547e4a-ae81-4e97-9a06-ea8bec54669d admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:48.222413 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-0d547e4a-ae81-4e97-9a06-ea8bec54669d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9b8cb43c41743a9a9e3aa1b28f78b61'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:48.223090 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-0d547e4a-ae81-4e97-9a06-ea8bec54669d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9b8cb43c41743a9a9e3aa1b28f78b61'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:48.296599 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-0d547e4a-ae81-4e97-9a06-ea8bec54669d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9b8cb43c41743a9a9e3aa1b28f78b61'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:48.308878 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-0d547e4a-ae81-4e97-9a06-ea8bec54669d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9b8cb43c41743a9a9e3aa1b28f78b61'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:48.309545 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-0d547e4a-ae81-4e97-9a06-ea8bec54669d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9b8cb43c41743a9a9e3aa1b28f78b61'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:48.321682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1093/2171] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:57:47 2022] DELETE /identity/v3/users/d9b8cb43c41743a9a9e3aa1b28f78b61 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:57:48.326695 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0468f-2cd6-43c0-9d50-6846d9d67a4d None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:48.326983 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0468f-2cd6-43c0-9d50-6846d9d67a4d None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:48.327260 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0468f-2cd6-43c0-9d50-6846d9d67a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:48.346715 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-37c0468f-2cd6-43c0-9d50-6846d9d67a4d None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:57:48.349710 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-37c0468f-2cd6-43c0-9d50-6846d9d67a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:48.395824 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:57:48.405249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1079/2172] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:57:48.414756 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d547e4a-ae81-4e97-9a06-ea8bec54669d admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:48.416161 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-0d547e4a-ae81-4e97-9a06-ea8bec54669d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:48.436402 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:57:48.473867 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d547e4a-ae81-4e97-9a06-ea8bec54669d admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:48.477551 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09105252-8237-4806-8f8a-49f0d494d665 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:48.478154 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09105252-8237-4806-8f8a-49f0d494d665 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:48.478433 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09105252-8237-4806-8f8a-49f0d494d665 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:48.478704 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09105252-8237-4806-8f8a-49f0d494d665 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:48.488365 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09105252-8237-4806-8f8a-49f0d494d665 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:48.489778 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09105252-8237-4806-8f8a-49f0d494d665 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:48.499139 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1094/2173] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:57:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:57:48.900314 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0468f-2cd6-43c0-9d50-6846d9d67a4d None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:48.901836 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-37c0468f-2cd6-43c0-9d50-6846d9d67a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:48.922937 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0468f-2cd6-43c0-9d50-6846d9d67a4d None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:48.928105 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364ee6a8-e059-4369-aa88-5a1de1cc2693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:48.928739 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ee6a8-e059-4369-aa88-5a1de1cc2693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:48.929020 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ee6a8-e059-4369-aa88-5a1de1cc2693 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:48.929287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364ee6a8-e059-4369-aa88-5a1de1cc2693 admin admin] PATH_INFO: `/v3/projects/2a34bf95f3044db19425033a38dbf5e2` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:48.930392 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ee6a8-e059-4369-aa88-5a1de1cc2693 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a34bf95f3044db19425033a38dbf5e2)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:48.932287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364ee6a8-e059-4369-aa88-5a1de1cc2693 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:49.044622 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-364ee6a8-e059-4369-aa88-5a1de1cc2693 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a34bf95f3044db19425033a38dbf5e2', 'request_id': 'req-364ee6a8-e059-4369-aa88-5a1de1cc2693'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:49.113396 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-364ee6a8-e059-4369-aa88-5a1de1cc2693 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a34bf95f3044db19425033a38dbf5e2', 'request_id': 'req-364ee6a8-e059-4369-aa88-5a1de1cc2693'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:49.118925 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-364ee6a8-e059-4369-aa88-5a1de1cc2693 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a34bf95f3044db19425033a38dbf5e2', 'request_id': 'req-364ee6a8-e059-4369-aa88-5a1de1cc2693'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:49.130723 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-364ee6a8-e059-4369-aa88-5a1de1cc2693 admin admin] The token cache is being invalidate because project 2a34bf95f3044db19425033a38dbf5e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:57:49.131076 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-364ee6a8-e059-4369-aa88-5a1de1cc2693 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:49.131700 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-364ee6a8-e059-4369-aa88-5a1de1cc2693 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:57:49.133203 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1080/2174] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:57:48 2022] DELETE /identity/v3/projects/2a34bf95f3044db19425033a38dbf5e2 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:57:49.138477 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2912596-3fce-4672-8249-61795bcba4bf None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:49.138946 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2912596-3fce-4672-8249-61795bcba4bf None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:49.140593 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2912596-3fce-4672-8249-61795bcba4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:49.158364 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-b2912596-3fce-4672-8249-61795bcba4bf None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:57:49.159480 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-b2912596-3fce-4672-8249-61795bcba4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:49.217248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1095/2175] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:57:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:57:49.226720 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bae3c-4db8-4094-ad25-d721761c16c2 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:49.227007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bae3c-4db8-4094-ad25-d721761c16c2 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:49.227284 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8bae3c-4db8-4094-ad25-d721761c16c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:49.247356 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-fa8bae3c-4db8-4094-ad25-d721761c16c2 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:57:49.249182 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-fa8bae3c-4db8-4094-ad25-d721761c16c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:49.257317 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1081/2176] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:57:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:57:49.261480 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2912596-3fce-4672-8249-61795bcba4bf None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:49.271207 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2912596-3fce-4672-8249-61795bcba4bf None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:49.275873 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860781cc-0e4d-4b79-ad11-701a7881d596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:49.276473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860781cc-0e4d-4b79-ad11-701a7881d596 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:49.276758 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860781cc-0e4d-4b79-ad11-701a7881d596 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:49.277034 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860781cc-0e4d-4b79-ad11-701a7881d596 admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:49.277677 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860781cc-0e4d-4b79-ad11-701a7881d596 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:57:49.277982 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860781cc-0e4d-4b79-ad11-701a7881d596 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:49.280121 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860781cc-0e4d-4b79-ad11-701a7881d596 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:49.291425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1096/2177] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:57:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:57:49.296137 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8bae3c-4db8-4094-ad25-d721761c16c2 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:49.305205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8bae3c-4db8-4094-ad25-d721761c16c2 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:49.309935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee4b64-001b-40d6-a85e-55faebd9934f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:49.310583 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee4b64-001b-40d6-a85e-55faebd9934f admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:49.310891 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee4b64-001b-40d6-a85e-55faebd9934f admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:49.311170 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee4b64-001b-40d6-a85e-55faebd9934f admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:49.311861 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee4b64-001b-40d6-a85e-55faebd9934f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:49.313775 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee4b64-001b-40d6-a85e-55faebd9934f admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:49.400020 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1082/2178] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:57:49 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:57:49.403975 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860781cc-0e4d-4b79-ad11-701a7881d596 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:49.414560 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860781cc-0e4d-4b79-ad11-701a7881d596 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:49.471142 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dfcf88-ee45-4485-a69d-51fc279bd313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:49.471791 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfcf88-ee45-4485-a69d-51fc279bd313 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:49.472077 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfcf88-ee45-4485-a69d-51fc279bd313 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:49.472351 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dfcf88-ee45-4485-a69d-51fc279bd313 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:49.472988 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dfcf88-ee45-4485-a69d-51fc279bd313 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:49.474929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dfcf88-ee45-4485-a69d-51fc279bd313 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:49.601908 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1097/2179] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:57:49 2022] POST /identity/v3/users => generated 287 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:57:49.605994 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee4b64-001b-40d6-a85e-55faebd9934f admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:49.617217 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee4b64-001b-40d6-a85e-55faebd9934f admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:49.621845 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ee091-14da-499b-a396-253ef6c6a543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:49.622443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ee091-14da-499b-a396-253ef6c6a543 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:49.622729 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ee091-14da-499b-a396-253ef6c6a543 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:49.623003 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722ee091-14da-499b-a396-253ef6c6a543 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:49.623755 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ee091-14da-499b-a396-253ef6c6a543 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:49.625371 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722ee091-14da-499b-a396-253ef6c6a543 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:49.635020 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1083/2180] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:57:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:57:49.638917 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dfcf88-ee45-4485-a69d-51fc279bd313 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:49.649463 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dfcf88-ee45-4485-a69d-51fc279bd313 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:49.654068 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d4d3a0-1d79-4a40-a773-ee8cb0527259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:49.654687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4d3a0-1d79-4a40-a773-ee8cb0527259 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:49.654966 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4d3a0-1d79-4a40-a773-ee8cb0527259 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:49.655240 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d4d3a0-1d79-4a40-a773-ee8cb0527259 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:49.656549 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d4d3a0-1d79-4a40-a773-ee8cb0527259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:49.658136 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d4d3a0-1d79-4a40-a773-ee8cb0527259 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:49.667904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1098/2181] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:57:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:57:49.671580 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ee091-14da-499b-a396-253ef6c6a543 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:49.681472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722ee091-14da-499b-a396-253ef6c6a543 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:49.685989 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed13bfa-df91-4862-884c-656a63011677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:49.686633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed13bfa-df91-4862-884c-656a63011677 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:49.686916 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed13bfa-df91-4862-884c-656a63011677 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:49.687190 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed13bfa-df91-4862-884c-656a63011677 admin admin] PATH_INFO: `/v3/projects/9c715bd8476f4fa3a77f2518388fec07/users/3f9022ea5895499c8e0c9757cc9a6c80/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:49.701755 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed13bfa-df91-4862-884c-656a63011677 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c715bd8476f4fa3a77f2518388fec07, user_id=3f9022ea5895499c8e0c9757cc9a6c80, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:49.704998 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed13bfa-df91-4862-884c-656a63011677 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:49.801735 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1084/2182] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:57:49 2022] PUT /identity/v3/projects/9c715bd8476f4fa3a77f2518388fec07/users/3f9022ea5895499c8e0c9757cc9a6c80/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:57:49.812024 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe4821-e43b-4229-868c-659e99d02a7c None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:49.812324 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe4821-e43b-4229-868c-659e99d02a7c None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:49.812621 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe4821-e43b-4229-868c-659e99d02a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:49.841611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-63fe4821-e43b-4229-868c-659e99d02a7c None None] MFA Rules not processed for user `3f9022ea5895499c8e0c9757cc9a6c80`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:57:49.842825 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-63fe4821-e43b-4229-868c-659e99d02a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:49.855722 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:57:49.907350 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1099/2183] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:57:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:57:49.911937 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed13bfa-df91-4862-884c-656a63011677 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:49.972560 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed13bfa-df91-4862-884c-656a63011677 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:49.977201 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f0779-ed82-4fe0-ae51-342bd2d4674c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:49.977798 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f0779-ed82-4fe0-ae51-342bd2d4674c admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:49.978079 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f0779-ed82-4fe0-ae51-342bd2d4674c admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:49.978353 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f0779-ed82-4fe0-ae51-342bd2d4674c admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:49.979001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f0779-ed82-4fe0-ae51-342bd2d4674c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:49.980912 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f0779-ed82-4fe0-ae51-342bd2d4674c admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:50.071080 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1085/2184] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:57:49 2022] POST /identity/v3/projects => generated 371 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:57:50.075106 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe4821-e43b-4229-868c-659e99d02a7c None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:50.086758 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe4821-e43b-4229-868c-659e99d02a7c None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:50.142832 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5620695-3e26-4569-bc88-f788fc56ea39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:50.143492 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5620695-3e26-4569-bc88-f788fc56ea39 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:50.143707 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5620695-3e26-4569-bc88-f788fc56ea39 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:50.144019 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5620695-3e26-4569-bc88-f788fc56ea39 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:50.144754 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5620695-3e26-4569-bc88-f788fc56ea39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:50.146663 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5620695-3e26-4569-bc88-f788fc56ea39 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:50.265810 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1100/2185] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:57:50 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:57:50.269628 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f0779-ed82-4fe0-ae51-342bd2d4674c admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:50.275005 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:57:50.280337 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f0779-ed82-4fe0-ae51-342bd2d4674c admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:50.285060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a2f04-e9f3-4ebb-a64f-1e36853c2b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:50.287544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a2f04-e9f3-4ebb-a64f-1e36853c2b2e admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:50.287544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a2f04-e9f3-4ebb-a64f-1e36853c2b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:50.287544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a2f04-e9f3-4ebb-a64f-1e36853c2b2e admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:50.287544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2a2f04-e9f3-4ebb-a64f-1e36853c2b2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:50.288512 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2a2f04-e9f3-4ebb-a64f-1e36853c2b2e admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:50.298917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1086/2186] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:57:50.302877 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5620695-3e26-4569-bc88-f788fc56ea39 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:50.313371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5620695-3e26-4569-bc88-f788fc56ea39 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:50.317943 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f882a58-6355-4e22-b8fc-80a3249f088e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:50.318599 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f882a58-6355-4e22-b8fc-80a3249f088e admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:50.318859 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f882a58-6355-4e22-b8fc-80a3249f088e admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:50.319128 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f882a58-6355-4e22-b8fc-80a3249f088e admin admin] PATH_INFO: `/v3/projects/f009e72b63144aeeac0d410685c70f14/users/a569369c25cb48728b0e82acb2cbac22/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:50.333997 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f882a58-6355-4e22-b8fc-80a3249f088e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f009e72b63144aeeac0d410685c70f14, user_id=a569369c25cb48728b0e82acb2cbac22, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:50.336948 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f882a58-6355-4e22-b8fc-80a3249f088e admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:50.407932 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1101/2187] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:57:50 2022] PUT /identity/v3/projects/f009e72b63144aeeac0d410685c70f14/users/a569369c25cb48728b0e82acb2cbac22/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:57:50.411896 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a2f04-e9f3-4ebb-a64f-1e36853c2b2e admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:50.422177 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a2f04-e9f3-4ebb-a64f-1e36853c2b2e admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:50.477019 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2b209-0341-48a3-9c24-4842dfa142d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:50.477624 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2b209-0341-48a3-9c24-4842dfa142d8 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:50.477907 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2b209-0341-48a3-9c24-4842dfa142d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:50.478295 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2b209-0341-48a3-9c24-4842dfa142d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:50.478856 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f2b209-0341-48a3-9c24-4842dfa142d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:50.480539 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f2b209-0341-48a3-9c24-4842dfa142d8 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:50.490661 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1087/2188] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:57:50.494802 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f882a58-6355-4e22-b8fc-80a3249f088e admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:50.505772 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f882a58-6355-4e22-b8fc-80a3249f088e admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:50.510466 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38646c16-10f9-41f1-a1e4-60838a7ceeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:50.511111 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38646c16-10f9-41f1-a1e4-60838a7ceeea admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:50.511390 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38646c16-10f9-41f1-a1e4-60838a7ceeea admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:50.511663 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38646c16-10f9-41f1-a1e4-60838a7ceeea admin admin] PATH_INFO: `/v3/projects/f009e72b63144aeeac0d410685c70f14/users/a569369c25cb48728b0e82acb2cbac22/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:50.513613 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38646c16-10f9-41f1-a1e4-60838a7ceeea admin admin] RBAC: Authorizing `identity:create_grant(project_id=f009e72b63144aeeac0d410685c70f14, user_id=a569369c25cb48728b0e82acb2cbac22, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:50.517345 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38646c16-10f9-41f1-a1e4-60838a7ceeea admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:50.526802 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1102/2189] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:57:50 2022] PUT /identity/v3/projects/f009e72b63144aeeac0d410685c70f14/users/a569369c25cb48728b0e82acb2cbac22/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:57:50.536859 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b8ec8-54d7-41af-b504-2cf1ccd90edd None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:50.537146 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b8ec8-54d7-41af-b504-2cf1ccd90edd None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:50.537423 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b8ec8-54d7-41af-b504-2cf1ccd90edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:50.567465 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-7c3b8ec8-54d7-41af-b504-2cf1ccd90edd None None] MFA Rules not processed for user `a569369c25cb48728b0e82acb2cbac22`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:57:50.568704 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-7c3b8ec8-54d7-41af-b504-2cf1ccd90edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:50.635251 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1088/2190] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:57:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:57:50.646681 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38646c16-10f9-41f1-a1e4-60838a7ceeea admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:50.648357 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-38646c16-10f9-41f1-a1e4-60838a7ceeea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:57:50.707919 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38646c16-10f9-41f1-a1e4-60838a7ceeea admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:50.711731 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ccfbcf-f5e1-43f0-83a5-77e2cd3a0a41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:50.712355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ccfbcf-f5e1-43f0-83a5-77e2cd3a0a41 service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:50.712628 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ccfbcf-f5e1-43f0-83a5-77e2cd3a0a41 service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:50.712891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ccfbcf-f5e1-43f0-83a5-77e2cd3a0a41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:50.722225 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ccfbcf-f5e1-43f0-83a5-77e2cd3a0a41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:50.723648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ccfbcf-f5e1-43f0-83a5-77e2cd3a0a41 service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:50.732897 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1103/2191] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:57:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:57:50.923272 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b8ec8-54d7-41af-b504-2cf1ccd90edd None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:57:50.932893 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b8ec8-54d7-41af-b504-2cf1ccd90edd None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:57:50.936370 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9398245-5973-4a28-84d7-26623336f7bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:57:50.936950 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9398245-5973-4a28-84d7-26623336f7bf service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:57:50.937226 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9398245-5973-4a28-84d7-26623336f7bf service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:57:50.937498 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9398245-5973-4a28-84d7-26623336f7bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:57:50.994745 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9398245-5973-4a28-84d7-26623336f7bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:57:50.996231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9398245-5973-4a28-84d7-26623336f7bf service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:57:51.005109 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1089/2192] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:57:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:58:00.728773 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=90,keystone:INSERT=5 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:01.001010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=95 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:02.729072 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ccfbcf-f5e1-43f0-83a5-77e2cd3a0a41 service cinder] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:02.730518 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-a7ccfbcf-f5e1-43f0-83a5-77e2cd3a0a41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:02.791806 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ccfbcf-f5e1-43f0-83a5-77e2cd3a0a41 service cinder] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:02.795621 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463d97b-f55e-41fb-acbe-2ca6bf753249 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:02.796297 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463d97b-f55e-41fb-acbe-2ca6bf753249 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:02.796597 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463d97b-f55e-41fb-acbe-2ca6bf753249 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:02.796870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463d97b-f55e-41fb-acbe-2ca6bf753249 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:02.798664 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6463d97b-f55e-41fb-acbe-2ca6bf753249 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:02.864681 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463d97b-f55e-41fb-acbe-2ca6bf753249 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:02.866103 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463d97b-f55e-41fb-acbe-2ca6bf753249 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:02.880266 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1104/2193] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:58:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:58:06.192534 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9398245-5973-4a28-84d7-26623336f7bf service cinder] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:06.194028 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c9398245-5973-4a28-84d7-26623336f7bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:06.265824 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9398245-5973-4a28-84d7-26623336f7bf service cinder] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:06.271548 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268d369-8622-4f7d-a2a7-a652b913c29e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b500a113a2aa4a95875d7e094b917db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:06.272243 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268d369-8622-4f7d-a2a7-a652b913c29e service placement] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:06.272528 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268d369-8622-4f7d-a2a7-a652b913c29e service placement] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:06.272798 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a268d369-8622-4f7d-a2a7-a652b913c29e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:06.274637 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-a268d369-8622-4f7d-a2a7-a652b913c29e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:06.286037 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268d369-8622-4f7d-a2a7-a652b913c29e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:06.287644 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a268d369-8622-4f7d-a2a7-a652b913c29e service placement] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:06.298443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1090/2194] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:58:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:58:06.331994 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463d97b-f55e-41fb-acbe-2ca6bf753249 service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:06.333440 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6463d97b-f55e-41fb-acbe-2ca6bf753249 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:06.346544 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463d97b-f55e-41fb-acbe-2ca6bf753249 service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:06.351255 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e8fbea-3526-4d4c-8d81-944acbe2b4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:06.351919 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8fbea-3526-4d4c-8d81-944acbe2b4ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:06.352199 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8fbea-3526-4d4c-8d81-944acbe2b4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:06.352473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8fbea-3526-4d4c-8d81-944acbe2b4ce admin admin] PATH_INFO: `/v3/users/57ee3a13e33549c9a90ebde353fcd6f2` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:06.353589 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e8fbea-3526-4d4c-8d81-944acbe2b4ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=57ee3a13e33549c9a90ebde353fcd6f2)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:06.355484 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e8fbea-3526-4d4c-8d81-944acbe2b4ce admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:06.534600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-50e8fbea-3526-4d4c-8d81-944acbe2b4ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57ee3a13e33549c9a90ebde353fcd6f2'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:06.535727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-50e8fbea-3526-4d4c-8d81-944acbe2b4ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57ee3a13e33549c9a90ebde353fcd6f2'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:06.616818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-50e8fbea-3526-4d4c-8d81-944acbe2b4ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57ee3a13e33549c9a90ebde353fcd6f2'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:06.630366 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-50e8fbea-3526-4d4c-8d81-944acbe2b4ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57ee3a13e33549c9a90ebde353fcd6f2'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:06.631052 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-50e8fbea-3526-4d4c-8d81-944acbe2b4ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57ee3a13e33549c9a90ebde353fcd6f2'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:06.643389 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1105/2195] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:58:06 2022] DELETE /identity/v3/users/57ee3a13e33549c9a90ebde353fcd6f2 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:06.870286 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268d369-8622-4f7d-a2a7-a652b913c29e service placement] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:06.882058 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a268d369-8622-4f7d-a2a7-a652b913c29e service placement] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:06.938162 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c6a2e-dc19-4815-8688-c93d6b79b240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:06.938793 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c6a2e-dc19-4815-8688-c93d6b79b240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:06.939078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c6a2e-dc19-4815-8688-c93d6b79b240 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:06.939355 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c6a2e-dc19-4815-8688-c93d6b79b240 admin admin] PATH_INFO: `/v3/projects/3ed4ecab9c234dd293ba0034d71d03f7` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:06.940498 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c6a2e-dc19-4815-8688-c93d6b79b240 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ed4ecab9c234dd293ba0034d71d03f7)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:06.943038 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c6a2e-dc19-4815-8688-c93d6b79b240 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:07.074675 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-460c6a2e-dc19-4815-8688-c93d6b79b240 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ed4ecab9c234dd293ba0034d71d03f7', 'request_id': 'req-460c6a2e-dc19-4815-8688-c93d6b79b240'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:07.148174 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-460c6a2e-dc19-4815-8688-c93d6b79b240 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ed4ecab9c234dd293ba0034d71d03f7', 'request_id': 'req-460c6a2e-dc19-4815-8688-c93d6b79b240'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:07.153884 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-460c6a2e-dc19-4815-8688-c93d6b79b240 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ed4ecab9c234dd293ba0034d71d03f7', 'request_id': 'req-460c6a2e-dc19-4815-8688-c93d6b79b240'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:07.165618 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-460c6a2e-dc19-4815-8688-c93d6b79b240 admin admin] The token cache is being invalidate because project 3ed4ecab9c234dd293ba0034d71d03f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:58:07.165908 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-460c6a2e-dc19-4815-8688-c93d6b79b240 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:07.166551 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-460c6a2e-dc19-4815-8688-c93d6b79b240 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:07.168046 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1091/2196] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:58:06 2022] DELETE /identity/v3/projects/3ed4ecab9c234dd293ba0034d71d03f7 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:07.173011 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9427958-a1d5-4a80-9c68-a6ca54b5ca06 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:07.173303 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9427958-a1d5-4a80-9c68-a6ca54b5ca06 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:07.173585 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9427958-a1d5-4a80-9c68-a6ca54b5ca06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:07.193635 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-b9427958-a1d5-4a80-9c68-a6ca54b5ca06 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:07.195394 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-b9427958-a1d5-4a80-9c68-a6ca54b5ca06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:07.213173 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:07.256692 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1106/2197] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:58:07.264512 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193326d8-de5c-4476-9c70-5917df34db88 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:07.264950 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193326d8-de5c-4476-9c70-5917df34db88 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:07.268277 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193326d8-de5c-4476-9c70-5917df34db88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:07.295762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-193326d8-de5c-4476-9c70-5917df34db88 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:07.295762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-193326d8-de5c-4476-9c70-5917df34db88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:07.307832 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1092/2198] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:58:07.307982 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9427958-a1d5-4a80-9c68-a6ca54b5ca06 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:07.314813 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9427958-a1d5-4a80-9c68-a6ca54b5ca06 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:07.320093 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3eaac-8230-4005-a5f0-61c5c88e7f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:07.320897 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3eaac-8230-4005-a5f0-61c5c88e7f7c admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:07.321337 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3eaac-8230-4005-a5f0-61c5c88e7f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:07.321715 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f3eaac-8230-4005-a5f0-61c5c88e7f7c admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:07.322480 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3eaac-8230-4005-a5f0-61c5c88e7f7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:58:07.322884 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3eaac-8230-4005-a5f0-61c5c88e7f7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:07.324800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f3eaac-8230-4005-a5f0-61c5c88e7f7c admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:07.337951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1107/2199] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:58:07.345767 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193326d8-de5c-4476-9c70-5917df34db88 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:07.356115 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193326d8-de5c-4476-9c70-5917df34db88 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:07.361284 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92dfec-ebd9-40cd-91d0-dffac92119bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:07.362034 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92dfec-ebd9-40cd-91d0-dffac92119bb admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:07.362412 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92dfec-ebd9-40cd-91d0-dffac92119bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:07.362752 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92dfec-ebd9-40cd-91d0-dffac92119bb admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:07.364498 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b92dfec-ebd9-40cd-91d0-dffac92119bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:07.366079 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b92dfec-ebd9-40cd-91d0-dffac92119bb admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:07.377755 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:07.460592 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1093/2200] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:58:07 2022] POST /identity/v3/projects => generated 405 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:58:07.464605 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3eaac-8230-4005-a5f0-61c5c88e7f7c admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:07.475187 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f3eaac-8230-4005-a5f0-61c5c88e7f7c admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:07.532093 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9486b5e-06c2-4983-9a42-98db70bd5d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:07.532678 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9486b5e-06c2-4983-9a42-98db70bd5d95 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:07.532959 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9486b5e-06c2-4983-9a42-98db70bd5d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:07.533234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9486b5e-06c2-4983-9a42-98db70bd5d95 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:07.533887 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9486b5e-06c2-4983-9a42-98db70bd5d95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:07.536446 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9486b5e-06c2-4983-9a42-98db70bd5d95 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:07.653811 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1108/2201] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:58:07 2022] POST /identity/v3/users => generated 303 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:58:07.657668 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92dfec-ebd9-40cd-91d0-dffac92119bb admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:07.668509 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92dfec-ebd9-40cd-91d0-dffac92119bb admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:07.673030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4057ad-4bab-461d-8ba3-56bf0b4374fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:07.673623 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4057ad-4bab-461d-8ba3-56bf0b4374fa admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:07.673902 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4057ad-4bab-461d-8ba3-56bf0b4374fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:07.674174 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4057ad-4bab-461d-8ba3-56bf0b4374fa admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:07.674825 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4057ad-4bab-461d-8ba3-56bf0b4374fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:07.677071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4057ad-4bab-461d-8ba3-56bf0b4374fa admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:07.691545 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1094/2202] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:58:07.692843 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9486b5e-06c2-4983-9a42-98db70bd5d95 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:07.703325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9486b5e-06c2-4983-9a42-98db70bd5d95 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:07.708667 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9d6bd-bbb7-4e1e-b27b-965235930ede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:07.709277 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9d6bd-bbb7-4e1e-b27b-965235930ede admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:07.709558 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9d6bd-bbb7-4e1e-b27b-965235930ede admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:07.709836 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9d6bd-bbb7-4e1e-b27b-965235930ede admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:07.710482 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9d6bd-bbb7-4e1e-b27b-965235930ede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:07.712457 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9d6bd-bbb7-4e1e-b27b-965235930ede admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:07.724837 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1109/2203] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:58:07.729085 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4057ad-4bab-461d-8ba3-56bf0b4374fa admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:07.738992 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4057ad-4bab-461d-8ba3-56bf0b4374fa admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:07.743521 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7de534-8f25-46d4-b431-e1f650a6daac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:07.744239 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7de534-8f25-46d4-b431-e1f650a6daac admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:07.744519 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7de534-8f25-46d4-b431-e1f650a6daac admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:07.744788 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7de534-8f25-46d4-b431-e1f650a6daac admin admin] PATH_INFO: `/v3/projects/04e21335ef1743fcb09d6e5625563329/users/6fbb60b0d33c425c9d6d7eb03f396c39/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:07.759114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7de534-8f25-46d4-b431-e1f650a6daac admin admin] RBAC: Authorizing `identity:create_grant(project_id=04e21335ef1743fcb09d6e5625563329, user_id=6fbb60b0d33c425c9d6d7eb03f396c39, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:07.761805 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7de534-8f25-46d4-b431-e1f650a6daac admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:07.870905 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1095/2204] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:58:07 2022] PUT /identity/v3/projects/04e21335ef1743fcb09d6e5625563329/users/6fbb60b0d33c425c9d6d7eb03f396c39/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:07.875907 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded1e17-0137-4d57-9176-aa0c382ed792 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:07.876196 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded1e17-0137-4d57-9176-aa0c382ed792 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:07.876473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded1e17-0137-4d57-9176-aa0c382ed792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:07.897029 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-8ded1e17-0137-4d57-9176-aa0c382ed792 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:07.898375 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-8ded1e17-0137-4d57-9176-aa0c382ed792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:07.961311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1110/2205] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:58:07.971926 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7de534-8f25-46d4-b431-e1f650a6daac admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:07.975675 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-8a7de534-8f25-46d4-b431-e1f650a6daac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:08.033737 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7de534-8f25-46d4-b431-e1f650a6daac admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:08.038094 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa29c29-31b2-43db-b2f2-dab648db83a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:08.038700 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa29c29-31b2-43db-b2f2-dab648db83a8 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:08.038983 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa29c29-31b2-43db-b2f2-dab648db83a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:08.039254 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa29c29-31b2-43db-b2f2-dab648db83a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:08.048808 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa29c29-31b2-43db-b2f2-dab648db83a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:08.050299 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa29c29-31b2-43db-b2f2-dab648db83a8 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:08.060550 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1096/2206] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:58:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:58:09.036971 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded1e17-0137-4d57-9176-aa0c382ed792 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:09.038475 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-8ded1e17-0137-4d57-9176-aa0c382ed792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:09.075546 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:09.125127 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded1e17-0137-4d57-9176-aa0c382ed792 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:09.133437 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639d2fd-d626-45ba-942b-3f567ae0be81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:09.134076 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639d2fd-d626-45ba-942b-3f567ae0be81 service nova] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:09.134479 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639d2fd-d626-45ba-942b-3f567ae0be81 service nova] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:09.134776 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639d2fd-d626-45ba-942b-3f567ae0be81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:09.136593 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-e639d2fd-d626-45ba-942b-3f567ae0be81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:09.201840 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639d2fd-d626-45ba-942b-3f567ae0be81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:09.203561 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639d2fd-d626-45ba-942b-3f567ae0be81 service nova] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:09.213742 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1111/2207] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:58:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:58:09.235692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc9b2b-6db6-43dc-a535-f6c27e8c8be5 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:09.237358 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc9b2b-6db6-43dc-a535-f6c27e8c8be5 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:09.237649 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc9b2b-6db6-43dc-a535-f6c27e8c8be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:09.273936 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-29dc9b2b-6db6-43dc-a535-f6c27e8c8be5 None None] MFA Rules not processed for user `6fbb60b0d33c425c9d6d7eb03f396c39`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:09.273936 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-29dc9b2b-6db6-43dc-a535-f6c27e8c8be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:09.360695 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1097/2208] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:09 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:58:09.365854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639d2fd-d626-45ba-942b-3f567ae0be81 service nova] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:09.376435 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639d2fd-d626-45ba-942b-3f567ae0be81 service nova] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:09.381869 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2a3ab-d92a-44a4-aff4-87290648df1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:09.382575 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2a3ab-d92a-44a4-aff4-87290648df1d admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:09.382966 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2a3ab-d92a-44a4-aff4-87290648df1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:09.383912 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2a3ab-d92a-44a4-aff4-87290648df1d admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:09.383912 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f2a3ab-d92a-44a4-aff4-87290648df1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:09.386221 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f2a3ab-d92a-44a4-aff4-87290648df1d admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:09.486302 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1112/2209] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:58:09 2022] POST /identity/v3/projects => generated 405 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:58:09.491179 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc9b2b-6db6-43dc-a535-f6c27e8c8be5 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:09.502384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc9b2b-6db6-43dc-a535-f6c27e8c8be5 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:09.548643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:09.560800 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e6382-917f-4106-8ec0-968f01c72e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:09.561408 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e6382-917f-4106-8ec0-968f01c72e92 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:09.561683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e6382-917f-4106-8ec0-968f01c72e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:09.561953 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e6382-917f-4106-8ec0-968f01c72e92 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:09.562598 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e6382-917f-4106-8ec0-968f01c72e92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:09.564548 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e6382-917f-4106-8ec0-968f01c72e92 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:09.693510 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1098/2210] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:58:09 2022] POST /identity/v3/users => generated 309 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:58:09.697685 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2a3ab-d92a-44a4-aff4-87290648df1d admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:09.708103 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2a3ab-d92a-44a4-aff4-87290648df1d admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:09.712945 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b2301-d922-4121-873e-08fb1f332d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:09.714425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b2301-d922-4121-873e-08fb1f332d6d admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:09.714425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b2301-d922-4121-873e-08fb1f332d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:09.714425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b2301-d922-4121-873e-08fb1f332d6d admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:09.714843 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78b2301-d922-4121-873e-08fb1f332d6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:09.716461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78b2301-d922-4121-873e-08fb1f332d6d admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:09.726842 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1113/2211] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:58:09.731127 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e6382-917f-4106-8ec0-968f01c72e92 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:09.743207 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e6382-917f-4106-8ec0-968f01c72e92 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:09.747662 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3b3b2-3bac-4887-b76c-e420ec0f4b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:09.748471 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3b3b2-3bac-4887-b76c-e420ec0f4b0a admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:09.748669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3b3b2-3bac-4887-b76c-e420ec0f4b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:09.748870 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a3b3b2-3bac-4887-b76c-e420ec0f4b0a admin admin] PATH_INFO: `/v3/projects/10876b91111d41bb8593e1f3adf1cf81/users/5d9c3265a753427cbdfb78f02a6f01be/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:09.764831 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a3b3b2-3bac-4887-b76c-e420ec0f4b0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=10876b91111d41bb8593e1f3adf1cf81, user_id=5d9c3265a753427cbdfb78f02a6f01be, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:09.767472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a3b3b2-3bac-4887-b76c-e420ec0f4b0a admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:09.871795 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1099/2212] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:58:09 2022] PUT /identity/v3/projects/10876b91111d41bb8593e1f3adf1cf81/users/5d9c3265a753427cbdfb78f02a6f01be/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:09.874872 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b2301-d922-4121-873e-08fb1f332d6d admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:09.886704 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b2301-d922-4121-873e-08fb1f332d6d admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:09.949041 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f13ae2-135e-44ed-8bbb-a0092ad0f654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:09.949659 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f13ae2-135e-44ed-8bbb-a0092ad0f654 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:09.949944 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f13ae2-135e-44ed-8bbb-a0092ad0f654 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:09.950221 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f13ae2-135e-44ed-8bbb-a0092ad0f654 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:09.950871 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f13ae2-135e-44ed-8bbb-a0092ad0f654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:09.952536 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f13ae2-135e-44ed-8bbb-a0092ad0f654 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:09.963179 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1114/2213] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:58:09.967311 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3b3b2-3bac-4887-b76c-e420ec0f4b0a admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:09.978578 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a3b3b2-3bac-4887-b76c-e420ec0f4b0a admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:09.983324 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27124d3e-0b70-49ca-81e5-d39d1d5327ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:09.983982 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27124d3e-0b70-49ca-81e5-d39d1d5327ae admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:09.984264 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27124d3e-0b70-49ca-81e5-d39d1d5327ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:09.984541 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27124d3e-0b70-49ca-81e5-d39d1d5327ae admin admin] PATH_INFO: `/v3/projects/10876b91111d41bb8593e1f3adf1cf81/users/5d9c3265a753427cbdfb78f02a6f01be/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:09.986491 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27124d3e-0b70-49ca-81e5-d39d1d5327ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=10876b91111d41bb8593e1f3adf1cf81, user_id=5d9c3265a753427cbdfb78f02a6f01be, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:09.989185 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27124d3e-0b70-49ca-81e5-d39d1d5327ae admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:10.000473 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1100/2214] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:58:09 2022] PUT /identity/v3/projects/10876b91111d41bb8593e1f3adf1cf81/users/5d9c3265a753427cbdfb78f02a6f01be/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:11.493079 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebf79b-2b7c-4e61-8f65-9cdec68af588 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:11.493647 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebf79b-2b7c-4e61-8f65-9cdec68af588 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:11.493647 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebf79b-2b7c-4e61-8f65-9cdec68af588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:11.523100 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-f4ebf79b-2b7c-4e61-8f65-9cdec68af588 None None] MFA Rules not processed for user `5d9c3265a753427cbdfb78f02a6f01be`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:11.524660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-f4ebf79b-2b7c-4e61-8f65-9cdec68af588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:11.543457 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:11.595127 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1115/2215] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:11 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:58:12.950832 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27124d3e-0b70-49ca-81e5-d39d1d5327ae admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:13.014014 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27124d3e-0b70-49ca-81e5-d39d1d5327ae admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:13.018517 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77972648-cb49-4313-8861-c6ecd65e0ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:13.019627 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77972648-cb49-4313-8861-c6ecd65e0ae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:13.019936 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77972648-cb49-4313-8861-c6ecd65e0ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:13.020204 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77972648-cb49-4313-8861-c6ecd65e0ae5 admin admin] PATH_INFO: `/v3/users/6fbb60b0d33c425c9d6d7eb03f396c39` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:13.021304 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77972648-cb49-4313-8861-c6ecd65e0ae5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fbb60b0d33c425c9d6d7eb03f396c39)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:13.023185 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77972648-cb49-4313-8861-c6ecd65e0ae5 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:13.235499 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-77972648-cb49-4313-8861-c6ecd65e0ae5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fbb60b0d33c425c9d6d7eb03f396c39'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:13.313501 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-77972648-cb49-4313-8861-c6ecd65e0ae5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fbb60b0d33c425c9d6d7eb03f396c39'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:13.329438 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-77972648-cb49-4313-8861-c6ecd65e0ae5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fbb60b0d33c425c9d6d7eb03f396c39'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:13.329438 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-77972648-cb49-4313-8861-c6ecd65e0ae5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fbb60b0d33c425c9d6d7eb03f396c39'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:13.329438 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-77972648-cb49-4313-8861-c6ecd65e0ae5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fbb60b0d33c425c9d6d7eb03f396c39'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:13.340987 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1101/2216] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:58:12 2022] DELETE /identity/v3/users/6fbb60b0d33c425c9d6d7eb03f396c39 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:13.595549 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ebf79b-2b7c-4e61-8f65-9cdec68af588 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:13.606312 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ebf79b-2b7c-4e61-8f65-9cdec68af588 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:13.663762 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a0a8f-00c9-44b9-b047-8d604c2b4b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:13.664414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a0a8f-00c9-44b9-b047-8d604c2b4b87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:13.664696 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a0a8f-00c9-44b9-b047-8d604c2b4b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:13.664970 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a0a8f-00c9-44b9-b047-8d604c2b4b87 admin admin] PATH_INFO: `/v3/projects/04e21335ef1743fcb09d6e5625563329` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:13.666074 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a0a8f-00c9-44b9-b047-8d604c2b4b87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04e21335ef1743fcb09d6e5625563329)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:13.668039 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379a0a8f-00c9-44b9-b047-8d604c2b4b87 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:13.800203 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-379a0a8f-00c9-44b9-b047-8d604c2b4b87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04e21335ef1743fcb09d6e5625563329', 'request_id': 'req-379a0a8f-00c9-44b9-b047-8d604c2b4b87'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:13.879307 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-379a0a8f-00c9-44b9-b047-8d604c2b4b87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04e21335ef1743fcb09d6e5625563329', 'request_id': 'req-379a0a8f-00c9-44b9-b047-8d604c2b4b87'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:13.885186 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-379a0a8f-00c9-44b9-b047-8d604c2b4b87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04e21335ef1743fcb09d6e5625563329', 'request_id': 'req-379a0a8f-00c9-44b9-b047-8d604c2b4b87'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:13.896492 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-379a0a8f-00c9-44b9-b047-8d604c2b4b87 admin admin] The token cache is being invalidate because project 04e21335ef1743fcb09d6e5625563329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:58:13.896781 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-379a0a8f-00c9-44b9-b047-8d604c2b4b87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:13.897408 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-379a0a8f-00c9-44b9-b047-8d604c2b4b87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:13.898972 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1116/2217] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:58:13 2022] DELETE /identity/v3/projects/04e21335ef1743fcb09d6e5625563329 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:13.902941 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77972648-cb49-4313-8861-c6ecd65e0ae5 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:13.904526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-77972648-cb49-4313-8861-c6ecd65e0ae5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:13.917822 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77972648-cb49-4313-8861-c6ecd65e0ae5 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:13.934699 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:13.975560 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f6dc06-946e-4144-8915-e1212658ed97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:13.978399 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6dc06-946e-4144-8915-e1212658ed97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:13.978399 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6dc06-946e-4144-8915-e1212658ed97 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:13.978399 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f6dc06-946e-4144-8915-e1212658ed97 admin admin] PATH_INFO: `/v3/users/5d9c3265a753427cbdfb78f02a6f01be` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:13.978399 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6dc06-946e-4144-8915-e1212658ed97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d9c3265a753427cbdfb78f02a6f01be)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:13.979841 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f6dc06-946e-4144-8915-e1212658ed97 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:14.185111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c3f6dc06-946e-4144-8915-e1212658ed97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d9c3265a753427cbdfb78f02a6f01be'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:14.304669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c3f6dc06-946e-4144-8915-e1212658ed97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d9c3265a753427cbdfb78f02a6f01be'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:14.317246 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c3f6dc06-946e-4144-8915-e1212658ed97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d9c3265a753427cbdfb78f02a6f01be'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:14.317882 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c3f6dc06-946e-4144-8915-e1212658ed97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d9c3265a753427cbdfb78f02a6f01be'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:14.318483 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c3f6dc06-946e-4144-8915-e1212658ed97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d9c3265a753427cbdfb78f02a6f01be'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:14.330437 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1102/2218] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:58:13 2022] DELETE /identity/v3/users/5d9c3265a753427cbdfb78f02a6f01be => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:14.569668 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a0a8f-00c9-44b9-b047-8d604c2b4b87 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:14.571826 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-379a0a8f-00c9-44b9-b047-8d604c2b4b87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:14.586589 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379a0a8f-00c9-44b9-b047-8d604c2b4b87 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:14.630751 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:14.644941 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adf6268-c3b0-4f04-b3f0-7d6c44d13253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:14.645575 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf6268-c3b0-4f04-b3f0-7d6c44d13253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:14.646419 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf6268-c3b0-4f04-b3f0-7d6c44d13253 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:14.646696 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adf6268-c3b0-4f04-b3f0-7d6c44d13253 admin admin] PATH_INFO: `/v3/projects/10876b91111d41bb8593e1f3adf1cf81` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:14.647843 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf6268-c3b0-4f04-b3f0-7d6c44d13253 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10876b91111d41bb8593e1f3adf1cf81)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:14.649756 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adf6268-c3b0-4f04-b3f0-7d6c44d13253 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:14.753361 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-4adf6268-c3b0-4f04-b3f0-7d6c44d13253 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10876b91111d41bb8593e1f3adf1cf81', 'request_id': 'req-4adf6268-c3b0-4f04-b3f0-7d6c44d13253'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:14.827841 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-4adf6268-c3b0-4f04-b3f0-7d6c44d13253 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10876b91111d41bb8593e1f3adf1cf81', 'request_id': 'req-4adf6268-c3b0-4f04-b3f0-7d6c44d13253'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:14.834160 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-4adf6268-c3b0-4f04-b3f0-7d6c44d13253 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10876b91111d41bb8593e1f3adf1cf81', 'request_id': 'req-4adf6268-c3b0-4f04-b3f0-7d6c44d13253'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:14.845536 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-4adf6268-c3b0-4f04-b3f0-7d6c44d13253 admin admin] The token cache is being invalidate because project 10876b91111d41bb8593e1f3adf1cf81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:58:14.845839 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-4adf6268-c3b0-4f04-b3f0-7d6c44d13253 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:14.846491 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-4adf6268-c3b0-4f04-b3f0-7d6c44d13253 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:14.848016 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1117/2219] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:58:14 2022] DELETE /identity/v3/projects/10876b91111d41bb8593e1f3adf1cf81 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:14.852848 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226acee0-2f0d-4226-a0a9-9564c1917b74 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:14.853138 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226acee0-2f0d-4226-a0a9-9564c1917b74 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:14.853416 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226acee0-2f0d-4226-a0a9-9564c1917b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:14.872825 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-226acee0-2f0d-4226-a0a9-9564c1917b74 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:14.874023 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-226acee0-2f0d-4226-a0a9-9564c1917b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:14.931495 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1103/2220] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:58:14.939139 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809f8ea-67bf-4d52-8a98-afd3102df9d0 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:14.939417 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809f8ea-67bf-4d52-8a98-afd3102df9d0 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:14.939688 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9809f8ea-67bf-4d52-8a98-afd3102df9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:14.960025 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-9809f8ea-67bf-4d52-8a98-afd3102df9d0 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:14.961216 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-9809f8ea-67bf-4d52-8a98-afd3102df9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:14.969295 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1118/2221] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:58:14.973241 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226acee0-2f0d-4226-a0a9-9564c1917b74 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:14.983006 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226acee0-2f0d-4226-a0a9-9564c1917b74 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:14.987529 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dabd5f-9cf4-46bf-82e3-c33f6772fe49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:14.988168 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dabd5f-9cf4-46bf-82e3-c33f6772fe49 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:14.988447 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dabd5f-9cf4-46bf-82e3-c33f6772fe49 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:14.988717 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dabd5f-9cf4-46bf-82e3-c33f6772fe49 admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:14.989387 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dabd5f-9cf4-46bf-82e3-c33f6772fe49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:58:14.990175 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dabd5f-9cf4-46bf-82e3-c33f6772fe49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:14.991753 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dabd5f-9cf4-46bf-82e3-c33f6772fe49 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:15.004237 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1104/2222] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:58:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:58:15.009245 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809f8ea-67bf-4d52-8a98-afd3102df9d0 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:15.018276 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9809f8ea-67bf-4d52-8a98-afd3102df9d0 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:15.022914 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16171ad-ba06-4572-b00f-95544c0bdebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:15.023509 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16171ad-ba06-4572-b00f-95544c0bdebb admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:15.023827 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16171ad-ba06-4572-b00f-95544c0bdebb admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:15.024101 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16171ad-ba06-4572-b00f-95544c0bdebb admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:15.024735 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16171ad-ba06-4572-b00f-95544c0bdebb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:15.026598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16171ad-ba06-4572-b00f-95544c0bdebb admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:15.111037 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1119/2223] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:58:15 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:58:15.115865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dabd5f-9cf4-46bf-82e3-c33f6772fe49 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:15.125837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dabd5f-9cf4-46bf-82e3-c33f6772fe49 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:15.155182 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:15.187236 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15189843-3444-4e90-b85d-53641c7330c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:15.187887 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15189843-3444-4e90-b85d-53641c7330c0 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:15.188171 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15189843-3444-4e90-b85d-53641c7330c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:15.188446 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15189843-3444-4e90-b85d-53641c7330c0 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:15.189069 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15189843-3444-4e90-b85d-53641c7330c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:15.191353 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15189843-3444-4e90-b85d-53641c7330c0 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:15.310564 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1105/2224] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:58:15 2022] POST /identity/v3/users => generated 289 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:58:15.314665 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16171ad-ba06-4572-b00f-95544c0bdebb admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:15.325569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16171ad-ba06-4572-b00f-95544c0bdebb admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:15.330061 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e9f44-ea89-4bdf-99ae-30493deda5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:15.330653 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e9f44-ea89-4bdf-99ae-30493deda5e5 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:15.330927 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e9f44-ea89-4bdf-99ae-30493deda5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:15.331196 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e9f44-ea89-4bdf-99ae-30493deda5e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:15.331873 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e9f44-ea89-4bdf-99ae-30493deda5e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:15.333674 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e9f44-ea89-4bdf-99ae-30493deda5e5 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:15.343717 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1120/2225] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:58:15.347764 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15189843-3444-4e90-b85d-53641c7330c0 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:15.358207 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15189843-3444-4e90-b85d-53641c7330c0 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:15.362743 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87498f50-5407-46ec-9329-c390ebe5e517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:15.363415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87498f50-5407-46ec-9329-c390ebe5e517 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:15.363721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87498f50-5407-46ec-9329-c390ebe5e517 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:15.364043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87498f50-5407-46ec-9329-c390ebe5e517 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:15.365243 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87498f50-5407-46ec-9329-c390ebe5e517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:15.366804 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87498f50-5407-46ec-9329-c390ebe5e517 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:15.376396 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1106/2226] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:58:15.380317 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e9f44-ea89-4bdf-99ae-30493deda5e5 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:15.390128 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e9f44-ea89-4bdf-99ae-30493deda5e5 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:15.394648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600dedb4-b278-4449-b8ed-5aaad6cb616d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:15.395282 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600dedb4-b278-4449-b8ed-5aaad6cb616d admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:15.395566 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600dedb4-b278-4449-b8ed-5aaad6cb616d admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:15.395870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600dedb4-b278-4449-b8ed-5aaad6cb616d admin admin] PATH_INFO: `/v3/projects/331186236c7b4176b15ba4579f80839a/users/fa3a3e33101743f7a3ec53b07b21663c/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:15.410125 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600dedb4-b278-4449-b8ed-5aaad6cb616d admin admin] RBAC: Authorizing `identity:create_grant(project_id=331186236c7b4176b15ba4579f80839a, user_id=fa3a3e33101743f7a3ec53b07b21663c, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:15.413275 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600dedb4-b278-4449-b8ed-5aaad6cb616d admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:15.487458 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1121/2227] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:58:15 2022] PUT /identity/v3/projects/331186236c7b4176b15ba4579f80839a/users/fa3a3e33101743f7a3ec53b07b21663c/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:58:15.499588 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb0e31-62ed-47a2-9817-ad0974977572 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:15.499912 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb0e31-62ed-47a2-9817-ad0974977572 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:15.500189 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb0e31-62ed-47a2-9817-ad0974977572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:15.528307 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-3bdb0e31-62ed-47a2-9817-ad0974977572 None None] MFA Rules not processed for user `fa3a3e33101743f7a3ec53b07b21663c`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:15.529437 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-3bdb0e31-62ed-47a2-9817-ad0974977572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:15.592160 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1107/2228] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:58:15.596578 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600dedb4-b278-4449-b8ed-5aaad6cb616d admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:15.656511 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600dedb4-b278-4449-b8ed-5aaad6cb616d admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:15.661210 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f71a4-075c-4573-a3d4-06c815bc4014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:15.662297 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f71a4-075c-4573-a3d4-06c815bc4014 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:15.662579 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f71a4-075c-4573-a3d4-06c815bc4014 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:15.662851 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740f71a4-075c-4573-a3d4-06c815bc4014 admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:15.663480 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740f71a4-075c-4573-a3d4-06c815bc4014 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:15.665401 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740f71a4-075c-4573-a3d4-06c815bc4014 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:15.680676 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:15.757715 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1122/2229] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:58:15 2022] POST /identity/v3/projects => generated 377 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:58:15.761845 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb0e31-62ed-47a2-9817-ad0974977572 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:15.773128 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb0e31-62ed-47a2-9817-ad0974977572 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:15.831432 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70478bb-654d-4986-9ec0-f3e4673cf5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:15.832079 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70478bb-654d-4986-9ec0-f3e4673cf5d8 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:15.832364 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70478bb-654d-4986-9ec0-f3e4673cf5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:15.832635 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70478bb-654d-4986-9ec0-f3e4673cf5d8 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:15.833761 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70478bb-654d-4986-9ec0-f3e4673cf5d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:15.835649 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70478bb-654d-4986-9ec0-f3e4673cf5d8 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:15.945430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1108/2230] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:58:15 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:58:15.949401 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f71a4-075c-4573-a3d4-06c815bc4014 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:15.960231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740f71a4-075c-4573-a3d4-06c815bc4014 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:15.965115 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2c6c3-f206-416e-98b9-c2715503b148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:15.965695 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2c6c3-f206-416e-98b9-c2715503b148 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:15.965975 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2c6c3-f206-416e-98b9-c2715503b148 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:15.966248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2c6c3-f206-416e-98b9-c2715503b148 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:15.966882 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c2c6c3-f206-416e-98b9-c2715503b148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:15.968507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c2c6c3-f206-416e-98b9-c2715503b148 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:15.979417 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1123/2231] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:58:15.983440 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70478bb-654d-4986-9ec0-f3e4673cf5d8 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:15.993767 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70478bb-654d-4986-9ec0-f3e4673cf5d8 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:15.998546 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edba94b-a460-432b-ad56-7ee6911da8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:16.001057 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edba94b-a460-432b-ad56-7ee6911da8bb admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:16.001057 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edba94b-a460-432b-ad56-7ee6911da8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:16.001057 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edba94b-a460-432b-ad56-7ee6911da8bb admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:16.001057 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edba94b-a460-432b-ad56-7ee6911da8bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:16.001929 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edba94b-a460-432b-ad56-7ee6911da8bb admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:16.012032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1109/2232] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:58:16.016073 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2c6c3-f206-416e-98b9-c2715503b148 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:16.026683 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2c6c3-f206-416e-98b9-c2715503b148 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:16.031782 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000260c6-b4e0-476e-a521-acf1999d9659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:16.033058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000260c6-b4e0-476e-a521-acf1999d9659 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:16.033058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000260c6-b4e0-476e-a521-acf1999d9659 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:16.033058 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000260c6-b4e0-476e-a521-acf1999d9659 admin admin] PATH_INFO: `/v3/projects/b3cb42ed8f884a4485254490b3c6811c/users/68cc74ccd67546afae6c5ec192c5ae2c/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:16.048521 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000260c6-b4e0-476e-a521-acf1999d9659 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3cb42ed8f884a4485254490b3c6811c, user_id=68cc74ccd67546afae6c5ec192c5ae2c, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:16.051219 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000260c6-b4e0-476e-a521-acf1999d9659 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:16.128015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1124/2233] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:58:16 2022] PUT /identity/v3/projects/b3cb42ed8f884a4485254490b3c6811c/users/68cc74ccd67546afae6c5ec192c5ae2c/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:16.138962 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7f401-4d26-48f1-a858-93c0537f25ca None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:16.139252 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7f401-4d26-48f1-a858-93c0537f25ca None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:16.139530 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c7f401-4d26-48f1-a858-93c0537f25ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:16.164995 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:16.167873 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-d2c7f401-4d26-48f1-a858-93c0537f25ca None None] MFA Rules not processed for user `68cc74ccd67546afae6c5ec192c5ae2c`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:16.169081 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-d2c7f401-4d26-48f1-a858-93c0537f25ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:16.232934 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1110/2234] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:58:16.237685 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000260c6-b4e0-476e-a521-acf1999d9659 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:16.299447 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000260c6-b4e0-476e-a521-acf1999d9659 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:16.304086 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f99a2-1d47-4a75-b5e1-a6a0b7b37831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:16.304688 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f99a2-1d47-4a75-b5e1-a6a0b7b37831 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:16.308363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f99a2-1d47-4a75-b5e1-a6a0b7b37831 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:16.308363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f99a2-1d47-4a75-b5e1-a6a0b7b37831 admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:16.308363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425f99a2-1d47-4a75-b5e1-a6a0b7b37831 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:16.308363 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425f99a2-1d47-4a75-b5e1-a6a0b7b37831 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:16.388907 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1125/2235] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:58:16 2022] POST /identity/v3/projects => generated 379 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:58:16.392959 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7f401-4d26-48f1-a858-93c0537f25ca None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:16.405135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c7f401-4d26-48f1-a858-93c0537f25ca None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:16.460343 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a234d7d-ece9-42a2-8e9f-58ef6b802576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:16.460952 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a234d7d-ece9-42a2-8e9f-58ef6b802576 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:16.461233 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a234d7d-ece9-42a2-8e9f-58ef6b802576 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:16.461509 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a234d7d-ece9-42a2-8e9f-58ef6b802576 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:16.462410 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a234d7d-ece9-42a2-8e9f-58ef6b802576 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:16.464407 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a234d7d-ece9-42a2-8e9f-58ef6b802576 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:16.572642 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1111/2236] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:58:16 2022] POST /identity/v3/users => generated 296 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:58:16.575983 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f99a2-1d47-4a75-b5e1-a6a0b7b37831 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:16.586302 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f99a2-1d47-4a75-b5e1-a6a0b7b37831 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:16.591888 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d41377-845a-437a-bf4f-5486d771f2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:16.592678 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d41377-845a-437a-bf4f-5486d771f2a0 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:16.593015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d41377-845a-437a-bf4f-5486d771f2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:16.593454 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d41377-845a-437a-bf4f-5486d771f2a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:16.594247 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d41377-845a-437a-bf4f-5486d771f2a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:16.596072 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d41377-845a-437a-bf4f-5486d771f2a0 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:16.607162 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1126/2237] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:58:16.611233 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a234d7d-ece9-42a2-8e9f-58ef6b802576 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:16.621728 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a234d7d-ece9-42a2-8e9f-58ef6b802576 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:16.626295 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5356873-f63e-434f-bf0f-02e97541c8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:16.626946 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5356873-f63e-434f-bf0f-02e97541c8be admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:16.627228 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5356873-f63e-434f-bf0f-02e97541c8be admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:16.627504 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5356873-f63e-434f-bf0f-02e97541c8be admin admin] PATH_INFO: `/v3/projects/57c0ca0bf9c34efd8ec556c98694c73e/users/01b5ebafde964672aef1b8251f1c3734/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:16.642806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5356873-f63e-434f-bf0f-02e97541c8be admin admin] RBAC: Authorizing `identity:create_grant(project_id=57c0ca0bf9c34efd8ec556c98694c73e, user_id=01b5ebafde964672aef1b8251f1c3734, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:16.645491 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5356873-f63e-434f-bf0f-02e97541c8be admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:16.708476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1112/2238] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:58:16 2022] PUT /identity/v3/projects/57c0ca0bf9c34efd8ec556c98694c73e/users/01b5ebafde964672aef1b8251f1c3734/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:16.712705 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d41377-845a-437a-bf4f-5486d771f2a0 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:16.722765 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d41377-845a-437a-bf4f-5486d771f2a0 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:16.778800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75f15c-6dcc-49a8-ae35-693be7d181ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:16.779404 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75f15c-6dcc-49a8-ae35-693be7d181ce admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:16.779689 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75f15c-6dcc-49a8-ae35-693be7d181ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:16.779993 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75f15c-6dcc-49a8-ae35-693be7d181ce admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:16.780632 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75f15c-6dcc-49a8-ae35-693be7d181ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:16.782203 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75f15c-6dcc-49a8-ae35-693be7d181ce admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:16.791871 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1127/2239] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:16 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:58:16.795871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5356873-f63e-434f-bf0f-02e97541c8be admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:16.806420 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5356873-f63e-434f-bf0f-02e97541c8be admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:16.810914 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b28ad8-52b9-41f4-897b-18ae3d2711fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:16.811552 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b28ad8-52b9-41f4-897b-18ae3d2711fb admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:16.811866 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b28ad8-52b9-41f4-897b-18ae3d2711fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:16.812141 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b28ad8-52b9-41f4-897b-18ae3d2711fb admin admin] PATH_INFO: `/v3/projects/57c0ca0bf9c34efd8ec556c98694c73e/users/01b5ebafde964672aef1b8251f1c3734/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:16.814041 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b28ad8-52b9-41f4-897b-18ae3d2711fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=57c0ca0bf9c34efd8ec556c98694c73e, user_id=01b5ebafde964672aef1b8251f1c3734, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:16.816689 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b28ad8-52b9-41f4-897b-18ae3d2711fb admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:16.827527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1113/2240] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:58:16 2022] PUT /identity/v3/projects/57c0ca0bf9c34efd8ec556c98694c73e/users/01b5ebafde964672aef1b8251f1c3734/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:16.837594 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56352601-6172-4b8e-b80b-fcdf85c81540 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:16.837884 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56352601-6172-4b8e-b80b-fcdf85c81540 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:16.838162 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56352601-6172-4b8e-b80b-fcdf85c81540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:16.866041 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-56352601-6172-4b8e-b80b-fcdf85c81540 None None] MFA Rules not processed for user `01b5ebafde964672aef1b8251f1c3734`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:16.867242 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-56352601-6172-4b8e-b80b-fcdf85c81540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:16.885451 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:16.933842 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1128/2241] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:16 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:58:16.945473 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b28ad8-52b9-41f4-897b-18ae3d2711fb admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:16.947160 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-d9b28ad8-52b9-41f4-897b-18ae3d2711fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:17.004659 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b28ad8-52b9-41f4-897b-18ae3d2711fb admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:17.008329 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b21396-9123-477c-8512-58ec04e54043 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:17.008929 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b21396-9123-477c-8512-58ec04e54043 service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:17.009209 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b21396-9123-477c-8512-58ec04e54043 service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:17.009482 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b21396-9123-477c-8512-58ec04e54043 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:17.018241 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b21396-9123-477c-8512-58ec04e54043 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:17.019644 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b21396-9123-477c-8512-58ec04e54043 service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:17.023461 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:17.029358 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1114/2242] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:58:16 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:58:17.128300 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56352601-6172-4b8e-b80b-fcdf85c81540 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:17.147196 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56352601-6172-4b8e-b80b-fcdf85c81540 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:17.147196 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc80c6e-2b09-4f86-82b3-44859288ee79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:17.147196 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc80c6e-2b09-4f86-82b3-44859288ee79 service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:17.147196 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc80c6e-2b09-4f86-82b3-44859288ee79 service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:17.152376 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc80c6e-2b09-4f86-82b3-44859288ee79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:17.200130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc80c6e-2b09-4f86-82b3-44859288ee79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:17.201553 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc80c6e-2b09-4f86-82b3-44859288ee79 service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:17.211019 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1129/2243] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:58:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:58:23.405653 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b21396-9123-477c-8512-58ec04e54043 service cinder] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:23.467393 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b21396-9123-477c-8512-58ec04e54043 service cinder] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:23.471982 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95305c31-3c4d-4af4-b357-fb8ac7fd1210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:23.472611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95305c31-3c4d-4af4-b357-fb8ac7fd1210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:23.472892 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95305c31-3c4d-4af4-b357-fb8ac7fd1210 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:23.473155 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95305c31-3c4d-4af4-b357-fb8ac7fd1210 admin admin] PATH_INFO: `/v3/users/fa3a3e33101743f7a3ec53b07b21663c` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:23.474274 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95305c31-3c4d-4af4-b357-fb8ac7fd1210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa3a3e33101743f7a3ec53b07b21663c)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:23.476755 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95305c31-3c4d-4af4-b357-fb8ac7fd1210 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:23.666552 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-95305c31-3c4d-4af4-b357-fb8ac7fd1210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa3a3e33101743f7a3ec53b07b21663c'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:23.745048 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-95305c31-3c4d-4af4-b357-fb8ac7fd1210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa3a3e33101743f7a3ec53b07b21663c'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:23.762999 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-95305c31-3c4d-4af4-b357-fb8ac7fd1210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa3a3e33101743f7a3ec53b07b21663c'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:23.765903 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-95305c31-3c4d-4af4-b357-fb8ac7fd1210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa3a3e33101743f7a3ec53b07b21663c'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:23.765903 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-95305c31-3c4d-4af4-b357-fb8ac7fd1210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa3a3e33101743f7a3ec53b07b21663c'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:23.781942 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1115/2244] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:58:23 2022] DELETE /identity/v3/users/fa3a3e33101743f7a3ec53b07b21663c => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:23.783944 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23cae1-ede5-4ea8-8f2b-16cc16301256 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:23.784397 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23cae1-ede5-4ea8-8f2b-16cc16301256 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:23.784707 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23cae1-ede5-4ea8-8f2b-16cc16301256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:23.804405 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-2e23cae1-ede5-4ea8-8f2b-16cc16301256 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:23.805651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-2e23cae1-ede5-4ea8-8f2b-16cc16301256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:23.866381 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1130/2245] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:58:23.876113 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95305c31-3c4d-4af4-b357-fb8ac7fd1210 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:23.877498 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-95305c31-3c4d-4af4-b357-fb8ac7fd1210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:23.939123 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95305c31-3c4d-4af4-b357-fb8ac7fd1210 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:23.942751 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4d30b-34b3-42ec-99f2-8f1dbef93879 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:23.943910 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4d30b-34b3-42ec-99f2-8f1dbef93879 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:23.944191 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4d30b-34b3-42ec-99f2-8f1dbef93879 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:23.944463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4d30b-34b3-42ec-99f2-8f1dbef93879 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:23.953457 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f4d30b-34b3-42ec-99f2-8f1dbef93879 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:23.954859 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f4d30b-34b3-42ec-99f2-8f1dbef93879 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:23.965226 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1116/2246] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:58:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:58:24.397758 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23cae1-ede5-4ea8-8f2b-16cc16301256 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:24.399337 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-2e23cae1-ede5-4ea8-8f2b-16cc16301256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:24.412517 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23cae1-ede5-4ea8-8f2b-16cc16301256 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:24.417554 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedc484-d35b-4cf0-bafe-be75058b42e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:24.418299 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedc484-d35b-4cf0-bafe-be75058b42e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:24.418538 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedc484-d35b-4cf0-bafe-be75058b42e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:24.418958 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedc484-d35b-4cf0-bafe-be75058b42e4 admin admin] PATH_INFO: `/v3/projects/331186236c7b4176b15ba4579f80839a` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:24.420804 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dedc484-d35b-4cf0-bafe-be75058b42e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=331186236c7b4176b15ba4579f80839a)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:24.423448 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dedc484-d35b-4cf0-bafe-be75058b42e4 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:24.558616 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-0dedc484-d35b-4cf0-bafe-be75058b42e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '331186236c7b4176b15ba4579f80839a', 'request_id': 'req-0dedc484-d35b-4cf0-bafe-be75058b42e4'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:24.560682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:24.634530 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-0dedc484-d35b-4cf0-bafe-be75058b42e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '331186236c7b4176b15ba4579f80839a', 'request_id': 'req-0dedc484-d35b-4cf0-bafe-be75058b42e4'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:24.640782 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-0dedc484-d35b-4cf0-bafe-be75058b42e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '331186236c7b4176b15ba4579f80839a', 'request_id': 'req-0dedc484-d35b-4cf0-bafe-be75058b42e4'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:24.652175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-0dedc484-d35b-4cf0-bafe-be75058b42e4 admin admin] The token cache is being invalidate because project 331186236c7b4176b15ba4579f80839a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:58:24.652462 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-0dedc484-d35b-4cf0-bafe-be75058b42e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:24.653100 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-0dedc484-d35b-4cf0-bafe-be75058b42e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:24.654596 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1131/2247] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:58:24 2022] DELETE /identity/v3/projects/331186236c7b4176b15ba4579f80839a => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:24.658812 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4d30b-34b3-42ec-99f2-8f1dbef93879 service neutron] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:24.668973 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4d30b-34b3-42ec-99f2-8f1dbef93879 service neutron] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:24.686147 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:24.729695 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92432c-b462-40ec-92f7-d677e900ddf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:24.730335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92432c-b462-40ec-92f7-d677e900ddf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:24.730622 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92432c-b462-40ec-92f7-d677e900ddf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:24.730897 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92432c-b462-40ec-92f7-d677e900ddf9 admin admin] PATH_INFO: `/v3/users/68cc74ccd67546afae6c5ec192c5ae2c` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:24.732117 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92432c-b462-40ec-92f7-d677e900ddf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68cc74ccd67546afae6c5ec192c5ae2c)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:24.734016 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92432c-b462-40ec-92f7-d677e900ddf9 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:24.935971 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-2b92432c-b462-40ec-92f7-d677e900ddf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68cc74ccd67546afae6c5ec192c5ae2c'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:25.041985 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-2b92432c-b462-40ec-92f7-d677e900ddf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68cc74ccd67546afae6c5ec192c5ae2c'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:25.054747 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-2b92432c-b462-40ec-92f7-d677e900ddf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68cc74ccd67546afae6c5ec192c5ae2c'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:25.055383 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-2b92432c-b462-40ec-92f7-d677e900ddf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68cc74ccd67546afae6c5ec192c5ae2c'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:25.056022 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-2b92432c-b462-40ec-92f7-d677e900ddf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68cc74ccd67546afae6c5ec192c5ae2c'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:25.068286 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1117/2248] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:58:24 2022] DELETE /identity/v3/users/68cc74ccd67546afae6c5ec192c5ae2c => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:25.492740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedc484-d35b-4cf0-bafe-be75058b42e4 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:25.492740 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-0dedc484-d35b-4cf0-bafe-be75058b42e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:25.506814 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedc484-d35b-4cf0-bafe-be75058b42e4 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:25.565679 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa336968-6283-4280-900d-b684bbd9ea76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:25.566319 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa336968-6283-4280-900d-b684bbd9ea76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:25.566604 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa336968-6283-4280-900d-b684bbd9ea76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:25.566893 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa336968-6283-4280-900d-b684bbd9ea76 admin admin] PATH_INFO: `/v3/projects/b3cb42ed8f884a4485254490b3c6811c` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:25.568105 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa336968-6283-4280-900d-b684bbd9ea76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3cb42ed8f884a4485254490b3c6811c)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:25.570625 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa336968-6283-4280-900d-b684bbd9ea76 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:25.696024 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-fa336968-6283-4280-900d-b684bbd9ea76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3cb42ed8f884a4485254490b3c6811c', 'request_id': 'req-fa336968-6283-4280-900d-b684bbd9ea76'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:25.753919 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-fa336968-6283-4280-900d-b684bbd9ea76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3cb42ed8f884a4485254490b3c6811c', 'request_id': 'req-fa336968-6283-4280-900d-b684bbd9ea76'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:25.761093 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-fa336968-6283-4280-900d-b684bbd9ea76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3cb42ed8f884a4485254490b3c6811c', 'request_id': 'req-fa336968-6283-4280-900d-b684bbd9ea76'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:25.772796 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-fa336968-6283-4280-900d-b684bbd9ea76 admin admin] The token cache is being invalidate because project b3cb42ed8f884a4485254490b3c6811c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:58:25.773089 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-fa336968-6283-4280-900d-b684bbd9ea76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:25.773752 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-fa336968-6283-4280-900d-b684bbd9ea76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:25.775236 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1132/2249] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:58:25 2022] DELETE /identity/v3/projects/b3cb42ed8f884a4485254490b3c6811c => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:25.779176 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92432c-b462-40ec-92f7-d677e900ddf9 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:25.780649 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-2b92432c-b462-40ec-92f7-d677e900ddf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:25.793654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92432c-b462-40ec-92f7-d677e900ddf9 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:25.847813 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e602c2-9a73-424d-b054-2fa882d5d722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:25.848430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e602c2-9a73-424d-b054-2fa882d5d722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:25.848706 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e602c2-9a73-424d-b054-2fa882d5d722 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:25.848973 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e602c2-9a73-424d-b054-2fa882d5d722 admin admin] PATH_INFO: `/v3/users/01b5ebafde964672aef1b8251f1c3734` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:25.850065 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e602c2-9a73-424d-b054-2fa882d5d722 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01b5ebafde964672aef1b8251f1c3734)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:25.852476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e602c2-9a73-424d-b054-2fa882d5d722 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:26.062618 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:26.067850 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-66e602c2-9a73-424d-b054-2fa882d5d722 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01b5ebafde964672aef1b8251f1c3734'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:26.143750 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-66e602c2-9a73-424d-b054-2fa882d5d722 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01b5ebafde964672aef1b8251f1c3734'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:26.157283 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-66e602c2-9a73-424d-b054-2fa882d5d722 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01b5ebafde964672aef1b8251f1c3734'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:26.158228 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-66e602c2-9a73-424d-b054-2fa882d5d722 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01b5ebafde964672aef1b8251f1c3734'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:26.158692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-66e602c2-9a73-424d-b054-2fa882d5d722 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01b5ebafde964672aef1b8251f1c3734'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:26.176374 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1118/2250] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:58:25 2022] DELETE /identity/v3/users/01b5ebafde964672aef1b8251f1c3734 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:26.626979 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa336968-6283-4280-900d-b684bbd9ea76 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:26.628689 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-fa336968-6283-4280-900d-b684bbd9ea76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:26.654997 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa336968-6283-4280-900d-b684bbd9ea76 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:26.721239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417699c2-f500-47be-bea9-181babf5bd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:26.721886 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417699c2-f500-47be-bea9-181babf5bd15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:26.722150 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417699c2-f500-47be-bea9-181babf5bd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:26.722419 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417699c2-f500-47be-bea9-181babf5bd15 admin admin] PATH_INFO: `/v3/projects/57c0ca0bf9c34efd8ec556c98694c73e` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:26.723542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417699c2-f500-47be-bea9-181babf5bd15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57c0ca0bf9c34efd8ec556c98694c73e)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:26.725525 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417699c2-f500-47be-bea9-181babf5bd15 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:26.736723 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:26.844452 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-417699c2-f500-47be-bea9-181babf5bd15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57c0ca0bf9c34efd8ec556c98694c73e', 'request_id': 'req-417699c2-f500-47be-bea9-181babf5bd15'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:26.909297 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-417699c2-f500-47be-bea9-181babf5bd15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57c0ca0bf9c34efd8ec556c98694c73e', 'request_id': 'req-417699c2-f500-47be-bea9-181babf5bd15'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:26.915970 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-417699c2-f500-47be-bea9-181babf5bd15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57c0ca0bf9c34efd8ec556c98694c73e', 'request_id': 'req-417699c2-f500-47be-bea9-181babf5bd15'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:26.927553 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-417699c2-f500-47be-bea9-181babf5bd15 admin admin] The token cache is being invalidate because project 57c0ca0bf9c34efd8ec556c98694c73e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:58:26.927870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-417699c2-f500-47be-bea9-181babf5bd15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:26.928527 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-417699c2-f500-47be-bea9-181babf5bd15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:26.930564 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1133/2251] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:58:26 2022] DELETE /identity/v3/projects/57c0ca0bf9c34efd8ec556c98694c73e => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:26.937323 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4527e16-3d85-4ffb-817c-037f3ad2d06a None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:26.937834 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4527e16-3d85-4ffb-817c-037f3ad2d06a None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:26.937946 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4527e16-3d85-4ffb-817c-037f3ad2d06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:26.959425 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-c4527e16-3d85-4ffb-817c-037f3ad2d06a None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:26.960709 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c4527e16-3d85-4ffb-817c-037f3ad2d06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:27.021885 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1119/2252] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:58:27.031312 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2269a-a3f4-49b6-877b-0a4609cbdc7b None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:27.031600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2269a-a3f4-49b6-877b-0a4609cbdc7b None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:27.032356 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2269a-a3f4-49b6-877b-0a4609cbdc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:27.051442 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-b4a2269a-a3f4-49b6-877b-0a4609cbdc7b None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:27.052672 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-b4a2269a-a3f4-49b6-877b-0a4609cbdc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:27.061275 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1134/2253] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:58:27.065542 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4527e16-3d85-4ffb-817c-037f3ad2d06a None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:27.076297 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4527e16-3d85-4ffb-817c-037f3ad2d06a None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:27.080881 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931dc98-6461-4905-8e9c-07e75684ee3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:27.081497 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931dc98-6461-4905-8e9c-07e75684ee3e admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:27.081772 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931dc98-6461-4905-8e9c-07e75684ee3e admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:27.082038 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931dc98-6461-4905-8e9c-07e75684ee3e admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:27.082683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931dc98-6461-4905-8e9c-07e75684ee3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:58:27.082983 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931dc98-6461-4905-8e9c-07e75684ee3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:27.084704 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931dc98-6461-4905-8e9c-07e75684ee3e admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:27.096532 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1120/2254] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:58:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:58:27.100910 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2269a-a3f4-49b6-877b-0a4609cbdc7b None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:27.110015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2269a-a3f4-49b6-877b-0a4609cbdc7b None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:27.114639 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed5a86-0499-4efe-9a0d-9a4facfb4543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:27.115232 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed5a86-0499-4efe-9a0d-9a4facfb4543 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:27.115532 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed5a86-0499-4efe-9a0d-9a4facfb4543 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:27.115821 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed5a86-0499-4efe-9a0d-9a4facfb4543 admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:27.116480 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed5a86-0499-4efe-9a0d-9a4facfb4543 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:27.118907 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed5a86-0499-4efe-9a0d-9a4facfb4543 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:27.229667 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1135/2255] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:58:27 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:58:27.233624 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931dc98-6461-4905-8e9c-07e75684ee3e admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:27.243909 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931dc98-6461-4905-8e9c-07e75684ee3e admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:27.300043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a297b01-6ff2-4ed1-ad0d-11c1f34329a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:27.300640 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a297b01-6ff2-4ed1-ad0d-11c1f34329a2 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:27.305798 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a297b01-6ff2-4ed1-ad0d-11c1f34329a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:27.305798 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a297b01-6ff2-4ed1-ad0d-11c1f34329a2 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:27.305798 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a297b01-6ff2-4ed1-ad0d-11c1f34329a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:27.305798 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a297b01-6ff2-4ed1-ad0d-11c1f34329a2 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:27.407286 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1121/2256] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:58:27 2022] POST /identity/v3/users => generated 287 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:58:27.411451 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed5a86-0499-4efe-9a0d-9a4facfb4543 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:27.422202 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed5a86-0499-4efe-9a0d-9a4facfb4543 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:27.426867 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897d307-2428-4950-840f-afd3fa85d769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:27.430872 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897d307-2428-4950-840f-afd3fa85d769 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:27.430872 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897d307-2428-4950-840f-afd3fa85d769 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:27.430872 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897d307-2428-4950-840f-afd3fa85d769 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:27.430872 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897d307-2428-4950-840f-afd3fa85d769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:27.430872 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897d307-2428-4950-840f-afd3fa85d769 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:27.440609 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1136/2257] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:58:27.444382 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a297b01-6ff2-4ed1-ad0d-11c1f34329a2 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:27.454075 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a297b01-6ff2-4ed1-ad0d-11c1f34329a2 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:27.458693 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09d2ba-c5fc-43bc-8c4f-833469887b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:27.459275 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09d2ba-c5fc-43bc-8c4f-833469887b41 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:27.459557 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09d2ba-c5fc-43bc-8c4f-833469887b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:27.459860 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09d2ba-c5fc-43bc-8c4f-833469887b41 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:27.460496 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09d2ba-c5fc-43bc-8c4f-833469887b41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:27.462295 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09d2ba-c5fc-43bc-8c4f-833469887b41 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:27.472689 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1122/2258] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:58:27.476325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897d307-2428-4950-840f-afd3fa85d769 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:27.486680 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897d307-2428-4950-840f-afd3fa85d769 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:27.491203 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a810fe-8034-4a73-8b6e-d24355d36d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:27.491892 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a810fe-8034-4a73-8b6e-d24355d36d1a admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:27.492174 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a810fe-8034-4a73-8b6e-d24355d36d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:27.492447 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a810fe-8034-4a73-8b6e-d24355d36d1a admin admin] PATH_INFO: `/v3/projects/b4af2d53a1544e8a99cf19d4f103fbf2/users/d20fa4b6873f478fa8cd133c669f8cd7/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:27.507213 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a810fe-8034-4a73-8b6e-d24355d36d1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4af2d53a1544e8a99cf19d4f103fbf2, user_id=d20fa4b6873f478fa8cd133c669f8cd7, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:27.509878 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a810fe-8034-4a73-8b6e-d24355d36d1a admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:27.572542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1137/2259] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:58:27 2022] PUT /identity/v3/projects/b4af2d53a1544e8a99cf19d4f103fbf2/users/d20fa4b6873f478fa8cd133c669f8cd7/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:58:27.577436 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295517f-6971-4ea0-8572-350e735ff071 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:27.577725 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295517f-6971-4ea0-8572-350e735ff071 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:27.577997 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2295517f-6971-4ea0-8572-350e735ff071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:27.597155 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-2295517f-6971-4ea0-8572-350e735ff071 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:27.598345 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-2295517f-6971-4ea0-8572-350e735ff071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:27.616465 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:27.655823 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1123/2260] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:58:27.665854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a810fe-8034-4a73-8b6e-d24355d36d1a admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:27.667618 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-69a810fe-8034-4a73-8b6e-d24355d36d1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:27.725119 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a810fe-8034-4a73-8b6e-d24355d36d1a admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:27.729418 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af2515-7929-4cbd-987b-4f0eba52732f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:27.730045 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af2515-7929-4cbd-987b-4f0eba52732f service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:27.730321 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af2515-7929-4cbd-987b-4f0eba52732f service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:27.730591 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af2515-7929-4cbd-987b-4f0eba52732f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:27.739528 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af2515-7929-4cbd-987b-4f0eba52732f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:27.740910 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af2515-7929-4cbd-987b-4f0eba52732f service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:27.750086 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1138/2261] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:58:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:58:29.645954 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd5cd4-89b7-49f8-b6d9-9a908617f1af None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:29.646384 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd5cd4-89b7-49f8-b6d9-9a908617f1af None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:29.646491 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd5cd4-89b7-49f8-b6d9-9a908617f1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:29.686022 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-c0cd5cd4-89b7-49f8-b6d9-9a908617f1af None None] MFA Rules not processed for user `d20fa4b6873f478fa8cd133c669f8cd7`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:29.687316 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c0cd5cd4-89b7-49f8-b6d9-9a908617f1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:29.843573 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1124/2262] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:58:29.848293 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af2515-7929-4cbd-987b-4f0eba52732f service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:29.868453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af2515-7929-4cbd-987b-4f0eba52732f service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:29.868453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139dc04-7812-43cc-b23b-d230a910fc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:29.868453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139dc04-7812-43cc-b23b-d230a910fc18 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:29.868453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139dc04-7812-43cc-b23b-d230a910fc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:29.868841 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139dc04-7812-43cc-b23b-d230a910fc18 admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:29.868841 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139dc04-7812-43cc-b23b-d230a910fc18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:29.869185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139dc04-7812-43cc-b23b-d230a910fc18 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:29.982643 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1139/2263] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:58:29 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:58:29.987033 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd5cd4-89b7-49f8-b6d9-9a908617f1af None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:29.999969 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd5cd4-89b7-49f8-b6d9-9a908617f1af None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:30.072955 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86f42c-2452-4a05-99f1-65530ecaf509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:30.073573 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86f42c-2452-4a05-99f1-65530ecaf509 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:30.073856 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86f42c-2452-4a05-99f1-65530ecaf509 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:30.074132 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86f42c-2452-4a05-99f1-65530ecaf509 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:30.074772 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad86f42c-2452-4a05-99f1-65530ecaf509 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:30.076839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad86f42c-2452-4a05-99f1-65530ecaf509 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:30.178772 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1125/2264] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:58:29 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:58:30.182670 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139dc04-7812-43cc-b23b-d230a910fc18 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:30.197148 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139dc04-7812-43cc-b23b-d230a910fc18 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:30.203053 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b1720-b902-4f44-bbb9-de556aef3198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:30.203770 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b1720-b902-4f44-bbb9-de556aef3198 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:30.204124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b1720-b902-4f44-bbb9-de556aef3198 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:30.204466 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b1720-b902-4f44-bbb9-de556aef3198 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:30.205178 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269b1720-b902-4f44-bbb9-de556aef3198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:30.206868 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269b1720-b902-4f44-bbb9-de556aef3198 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:30.219115 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1140/2265] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:58:30.223650 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86f42c-2452-4a05-99f1-65530ecaf509 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:30.233904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86f42c-2452-4a05-99f1-65530ecaf509 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:30.238621 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86d27e-1b78-4933-80ad-d64439e8e47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:30.239409 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86d27e-1b78-4933-80ad-d64439e8e47a admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:30.239789 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86d27e-1b78-4933-80ad-d64439e8e47a admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:30.240079 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86d27e-1b78-4933-80ad-d64439e8e47a admin admin] PATH_INFO: `/v3/projects/283acce91bf34bfdb5e0c79df63846fe/users/a76c572a801f4f6dac439ca4da0afbcd/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:30.259249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b86d27e-1b78-4933-80ad-d64439e8e47a admin admin] RBAC: Authorizing `identity:create_grant(project_id=283acce91bf34bfdb5e0c79df63846fe, user_id=a76c572a801f4f6dac439ca4da0afbcd, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:30.261940 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b86d27e-1b78-4933-80ad-d64439e8e47a admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:30.378842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1126/2266] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:58:30 2022] PUT /identity/v3/projects/283acce91bf34bfdb5e0c79df63846fe/users/a76c572a801f4f6dac439ca4da0afbcd/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:30.382671 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b1720-b902-4f44-bbb9-de556aef3198 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:30.398809 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:30.405528 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b1720-b902-4f44-bbb9-de556aef3198 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:30.470937 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e756037-774f-4628-9a14-5704013be568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:30.471674 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e756037-774f-4628-9a14-5704013be568 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:30.472043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e756037-774f-4628-9a14-5704013be568 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:30.472384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e756037-774f-4628-9a14-5704013be568 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:30.473227 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e756037-774f-4628-9a14-5704013be568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:30.475026 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e756037-774f-4628-9a14-5704013be568 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:30.487335 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1141/2267] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:58:30.491582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86d27e-1b78-4933-80ad-d64439e8e47a admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:30.504872 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86d27e-1b78-4933-80ad-d64439e8e47a admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:30.510976 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584d61b-855e-48bb-8329-e76736f3bce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:30.518777 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584d61b-855e-48bb-8329-e76736f3bce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:30.518777 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584d61b-855e-48bb-8329-e76736f3bce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:30.518777 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584d61b-855e-48bb-8329-e76736f3bce7 admin admin] PATH_INFO: `/v3/projects/283acce91bf34bfdb5e0c79df63846fe/users/a76c572a801f4f6dac439ca4da0afbcd/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:30.518777 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0584d61b-855e-48bb-8329-e76736f3bce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=283acce91bf34bfdb5e0c79df63846fe, user_id=a76c572a801f4f6dac439ca4da0afbcd, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:30.518777 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0584d61b-855e-48bb-8329-e76736f3bce7 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:30.533133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1127/2268] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:58:30 2022] PUT /identity/v3/projects/283acce91bf34bfdb5e0c79df63846fe/users/a76c572a801f4f6dac439ca4da0afbcd/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:58:31.985345 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d680cb6-9edc-4013-a5f4-386fc3e6799f None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:31.986543 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d680cb6-9edc-4013-a5f4-386fc3e6799f None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:31.987258 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d680cb6-9edc-4013-a5f4-386fc3e6799f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:32.025024 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-6d680cb6-9edc-4013-a5f4-386fc3e6799f None None] MFA Rules not processed for user `a76c572a801f4f6dac439ca4da0afbcd`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:32.026481 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6d680cb6-9edc-4013-a5f4-386fc3e6799f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:32.117037 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1142/2269] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:31 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:58:32.131915 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584d61b-855e-48bb-8329-e76736f3bce7 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:32.134102 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-0584d61b-855e-48bb-8329-e76736f3bce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:32.146088 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:32.197316 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584d61b-855e-48bb-8329-e76736f3bce7 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:32.200928 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76873ed-cfd4-48e0-913d-5ace5cb51bd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:32.201533 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76873ed-cfd4-48e0-913d-5ace5cb51bd2 service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:32.201810 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76873ed-cfd4-48e0-913d-5ace5cb51bd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:32.202080 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76873ed-cfd4-48e0-913d-5ace5cb51bd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:32.258586 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76873ed-cfd4-48e0-913d-5ace5cb51bd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:32.260038 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76873ed-cfd4-48e0-913d-5ace5cb51bd2 service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:32.270569 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1128/2270] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:58:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:58:36.113137 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d680cb6-9edc-4013-a5f4-386fc3e6799f None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:36.114586 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6d680cb6-9edc-4013-a5f4-386fc3e6799f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:36.126846 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d680cb6-9edc-4013-a5f4-386fc3e6799f None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:36.130494 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6c626-f0f6-44f1-ba5a-bda5a3a72c22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:36.131087 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6c626-f0f6-44f1-ba5a-bda5a3a72c22 service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:36.131366 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6c626-f0f6-44f1-ba5a-bda5a3a72c22 service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:36.131637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6c626-f0f6-44f1-ba5a-bda5a3a72c22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:36.141484 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb6c626-f0f6-44f1-ba5a-bda5a3a72c22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:36.147826 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb6c626-f0f6-44f1-ba5a-bda5a3a72c22 service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:36.152427 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1143/2271] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:58:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:58:38.776191 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76873ed-cfd4-48e0-913d-5ace5cb51bd2 service cinder] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:38.777514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-f76873ed-cfd4-48e0-913d-5ace5cb51bd2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:38.832178 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76873ed-cfd4-48e0-913d-5ace5cb51bd2 service cinder] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:38.835694 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b7bf9c-78d9-461d-b506-9d51df4cbe8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:38.836318 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7bf9c-78d9-461d-b506-9d51df4cbe8a service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:38.836594 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7bf9c-78d9-461d-b506-9d51df4cbe8a service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:38.837717 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b7bf9c-78d9-461d-b506-9d51df4cbe8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:38.839374 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-d9b7bf9c-78d9-461d-b506-9d51df4cbe8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:38.900101 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b7bf9c-78d9-461d-b506-9d51df4cbe8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:38.901560 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b7bf9c-78d9-461d-b506-9d51df4cbe8a service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:38.911394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1129/2272] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:58:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:58:41.570393 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6c626-f0f6-44f1-ba5a-bda5a3a72c22 service cinder] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:41.570393 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-2fb6c626-f0f6-44f1-ba5a-bda5a3a72c22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:41.585715 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6c626-f0f6-44f1-ba5a-bda5a3a72c22 service cinder] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:41.592139 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e238658-d345-45df-b25f-8122e2a40ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:41.592823 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e238658-d345-45df-b25f-8122e2a40ddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:41.593050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e238658-d345-45df-b25f-8122e2a40ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:41.593323 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e238658-d345-45df-b25f-8122e2a40ddf admin admin] PATH_INFO: `/v3/users/3f9022ea5895499c8e0c9757cc9a6c80` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:41.594588 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e238658-d345-45df-b25f-8122e2a40ddf admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f9022ea5895499c8e0c9757cc9a6c80)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:41.596573 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e238658-d345-45df-b25f-8122e2a40ddf admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:41.838154 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-6e238658-d345-45df-b25f-8122e2a40ddf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f9022ea5895499c8e0c9757cc9a6c80'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:41.839233 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-6e238658-d345-45df-b25f-8122e2a40ddf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f9022ea5895499c8e0c9757cc9a6c80'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:41.939758 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-6e238658-d345-45df-b25f-8122e2a40ddf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f9022ea5895499c8e0c9757cc9a6c80'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:41.951691 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:41.953965 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-6e238658-d345-45df-b25f-8122e2a40ddf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f9022ea5895499c8e0c9757cc9a6c80'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:41.954681 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-6e238658-d345-45df-b25f-8122e2a40ddf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f9022ea5895499c8e0c9757cc9a6c80'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:41.967141 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1144/2273] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:58:41 2022] DELETE /identity/v3/users/3f9022ea5895499c8e0c9757cc9a6c80 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:41.971656 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135085be-73f0-4035-8670-96bca67283d6 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:41.983067 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135085be-73f0-4035-8670-96bca67283d6 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:41.983067 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135085be-73f0-4035-8670-96bca67283d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:41.991687 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-135085be-73f0-4035-8670-96bca67283d6 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:41.992905 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-135085be-73f0-4035-8670-96bca67283d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:41.999131 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:42.062777 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1130/2274] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:58:42.074248 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e238658-d345-45df-b25f-8122e2a40ddf admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:42.075827 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6e238658-d345-45df-b25f-8122e2a40ddf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:42.150061 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e238658-d345-45df-b25f-8122e2a40ddf admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:42.154197 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03edd41-5b66-48de-9585-7473ee85aa19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:42.155084 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03edd41-5b66-48de-9585-7473ee85aa19 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:42.155361 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03edd41-5b66-48de-9585-7473ee85aa19 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:42.155993 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03edd41-5b66-48de-9585-7473ee85aa19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:42.168634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03edd41-5b66-48de-9585-7473ee85aa19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:42.172844 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03edd41-5b66-48de-9585-7473ee85aa19 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:42.184359 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1145/2275] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:58:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:58:42.786186 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135085be-73f0-4035-8670-96bca67283d6 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:42.787704 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-135085be-73f0-4035-8670-96bca67283d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:42.802181 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135085be-73f0-4035-8670-96bca67283d6 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:42.807360 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2273a0e-d3aa-4b73-9fb4-a5977b0366d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:42.808048 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2273a0e-d3aa-4b73-9fb4-a5977b0366d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:42.808449 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2273a0e-d3aa-4b73-9fb4-a5977b0366d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:42.808636 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2273a0e-d3aa-4b73-9fb4-a5977b0366d7 admin admin] PATH_INFO: `/v3/projects/9c715bd8476f4fa3a77f2518388fec07` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:42.810049 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2273a0e-d3aa-4b73-9fb4-a5977b0366d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c715bd8476f4fa3a77f2518388fec07)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:42.813820 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2273a0e-d3aa-4b73-9fb4-a5977b0366d7 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:42.835545 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03edd41-5b66-48de-9585-7473ee85aa19 service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:42.838140 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-b03edd41-5b66-48de-9585-7473ee85aa19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:42.852430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03edd41-5b66-48de-9585-7473ee85aa19 service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:42.858349 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a3d41-bf9f-47c9-b46d-9b6cde5ef200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:42.858982 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a3d41-bf9f-47c9-b46d-9b6cde5ef200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:42.859777 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a3d41-bf9f-47c9-b46d-9b6cde5ef200 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:42.859777 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484a3d41-bf9f-47c9-b46d-9b6cde5ef200 admin admin] PATH_INFO: `/v3/users/d1499b45cce347c39855da14d66e9690` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:42.860869 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a3d41-bf9f-47c9-b46d-9b6cde5ef200 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1499b45cce347c39855da14d66e9690)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:42.862823 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484a3d41-bf9f-47c9-b46d-9b6cde5ef200 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:42.934717 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c2273a0e-d3aa-4b73-9fb4-a5977b0366d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c715bd8476f4fa3a77f2518388fec07', 'request_id': 'req-c2273a0e-d3aa-4b73-9fb4-a5977b0366d7'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:42.999821 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c2273a0e-d3aa-4b73-9fb4-a5977b0366d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c715bd8476f4fa3a77f2518388fec07', 'request_id': 'req-c2273a0e-d3aa-4b73-9fb4-a5977b0366d7'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:43.007450 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c2273a0e-d3aa-4b73-9fb4-a5977b0366d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c715bd8476f4fa3a77f2518388fec07', 'request_id': 'req-c2273a0e-d3aa-4b73-9fb4-a5977b0366d7'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:43.019643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c2273a0e-d3aa-4b73-9fb4-a5977b0366d7 admin admin] The token cache is being invalidate because project 9c715bd8476f4fa3a77f2518388fec07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:58:43.019967 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c2273a0e-d3aa-4b73-9fb4-a5977b0366d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:43.020615 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-c2273a0e-d3aa-4b73-9fb4-a5977b0366d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:43.022122 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1131/2276] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:58:42 2022] DELETE /identity/v3/projects/9c715bd8476f4fa3a77f2518388fec07 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:43.026547 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2273a0e-d3aa-4b73-9fb4-a5977b0366d7 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:43.027861 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c2273a0e-d3aa-4b73-9fb4-a5977b0366d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:43.090238 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2273a0e-d3aa-4b73-9fb4-a5977b0366d7 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:43.101496 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34aa771-8087-4628-8a62-e6131699a0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:43.102203 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34aa771-8087-4628-8a62-e6131699a0ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:43.102430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34aa771-8087-4628-8a62-e6131699a0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:43.102691 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34aa771-8087-4628-8a62-e6131699a0ae admin admin] PATH_INFO: `/v3/users/a569369c25cb48728b0e82acb2cbac22` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:43.103929 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34aa771-8087-4628-8a62-e6131699a0ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=a569369c25cb48728b0e82acb2cbac22)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:43.105919 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34aa771-8087-4628-8a62-e6131699a0ae admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:43.106069 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-484a3d41-bf9f-47c9-b46d-9b6cde5ef200 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1499b45cce347c39855da14d66e9690'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:43.107912 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-484a3d41-bf9f-47c9-b46d-9b6cde5ef200 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1499b45cce347c39855da14d66e9690'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:43.202333 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-484a3d41-bf9f-47c9-b46d-9b6cde5ef200 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1499b45cce347c39855da14d66e9690'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:43.216149 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-484a3d41-bf9f-47c9-b46d-9b6cde5ef200 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1499b45cce347c39855da14d66e9690'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:43.216798 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-484a3d41-bf9f-47c9-b46d-9b6cde5ef200 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1499b45cce347c39855da14d66e9690'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:43.229709 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1146/2277] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:58:42 2022] DELETE /identity/v3/users/d1499b45cce347c39855da14d66e9690 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:58:43.298303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:43.314559 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f34aa771-8087-4628-8a62-e6131699a0ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a569369c25cb48728b0e82acb2cbac22'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:43.416972 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f34aa771-8087-4628-8a62-e6131699a0ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a569369c25cb48728b0e82acb2cbac22'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:43.429386 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f34aa771-8087-4628-8a62-e6131699a0ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a569369c25cb48728b0e82acb2cbac22'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:43.430058 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f34aa771-8087-4628-8a62-e6131699a0ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a569369c25cb48728b0e82acb2cbac22'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:43.431153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f34aa771-8087-4628-8a62-e6131699a0ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a569369c25cb48728b0e82acb2cbac22'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:43.443676 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1132/2278] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:58:43 2022] DELETE /identity/v3/users/a569369c25cb48728b0e82acb2cbac22 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:58:43.496138 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a3d41-bf9f-47c9-b46d-9b6cde5ef200 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:43.497566 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-484a3d41-bf9f-47c9-b46d-9b6cde5ef200 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:43.568153 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484a3d41-bf9f-47c9-b46d-9b6cde5ef200 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:43.572739 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:43.625267 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf14356-87b2-41c6-8828-e88d53f82bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:43.625903 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf14356-87b2-41c6-8828-e88d53f82bc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:43.632295 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf14356-87b2-41c6-8828-e88d53f82bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:43.632295 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf14356-87b2-41c6-8828-e88d53f82bc0 admin admin] PATH_INFO: `/v3/projects/a139d546eb6a4ff595a94d0e1528ee5d` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:43.632295 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf14356-87b2-41c6-8828-e88d53f82bc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a139d546eb6a4ff595a94d0e1528ee5d)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:43.632295 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf14356-87b2-41c6-8828-e88d53f82bc0 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:43.743687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-bbf14356-87b2-41c6-8828-e88d53f82bc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a139d546eb6a4ff595a94d0e1528ee5d', 'request_id': 'req-bbf14356-87b2-41c6-8828-e88d53f82bc0'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:43.858122 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-bbf14356-87b2-41c6-8828-e88d53f82bc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a139d546eb6a4ff595a94d0e1528ee5d', 'request_id': 'req-bbf14356-87b2-41c6-8828-e88d53f82bc0'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:43.864386 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34aa771-8087-4628-8a62-e6131699a0ae admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:43.866043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-bbf14356-87b2-41c6-8828-e88d53f82bc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a139d546eb6a4ff595a94d0e1528ee5d', 'request_id': 'req-bbf14356-87b2-41c6-8828-e88d53f82bc0'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:43.866312 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-f34aa771-8087-4628-8a62-e6131699a0ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:43.879701 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-bbf14356-87b2-41c6-8828-e88d53f82bc0 admin admin] The token cache is being invalidate because project a139d546eb6a4ff595a94d0e1528ee5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:58:43.880143 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-bbf14356-87b2-41c6-8828-e88d53f82bc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:43.880679 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-bbf14356-87b2-41c6-8828-e88d53f82bc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:43.880808 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34aa771-8087-4628-8a62-e6131699a0ae admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:43.882373 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1147/2279] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:58:43 2022] DELETE /identity/v3/projects/a139d546eb6a4ff595a94d0e1528ee5d => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:43.887038 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe826e-eed1-48e5-a454-9a2e6bd8ce0e None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:43.893824 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe826e-eed1-48e5-a454-9a2e6bd8ce0e None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:43.893824 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe826e-eed1-48e5-a454-9a2e6bd8ce0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:43.908739 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-5fbe826e-eed1-48e5-a454-9a2e6bd8ce0e None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:43.909938 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-5fbe826e-eed1-48e5-a454-9a2e6bd8ce0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:43.942608 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854b50e-69a0-4f9d-915a-ef5e95c48af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:43.943488 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854b50e-69a0-4f9d-915a-ef5e95c48af6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:43.943816 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854b50e-69a0-4f9d-915a-ef5e95c48af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:43.944127 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854b50e-69a0-4f9d-915a-ef5e95c48af6 admin admin] PATH_INFO: `/v3/projects/f009e72b63144aeeac0d410685c70f14` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:43.945348 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2854b50e-69a0-4f9d-915a-ef5e95c48af6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f009e72b63144aeeac0d410685c70f14)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:43.947315 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2854b50e-69a0-4f9d-915a-ef5e95c48af6 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:43.972905 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1148/2280] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:58:43.985483 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8c2ce-1a55-4db8-9cb9-fbc448a640bc None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:43.985483 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8c2ce-1a55-4db8-9cb9-fbc448a640bc None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:43.985483 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8c2ce-1a55-4db8-9cb9-fbc448a640bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:44.002474 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-6dc8c2ce-1a55-4db8-9cb9-fbc448a640bc None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:44.003670 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6dc8c2ce-1a55-4db8-9cb9-fbc448a640bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:44.011677 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1149/2281] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:58:44.015627 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8c2ce-1a55-4db8-9cb9-fbc448a640bc None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:44.024514 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc8c2ce-1a55-4db8-9cb9-fbc448a640bc None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:44.029227 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016de66-d667-48c7-9c43-f58f4d361ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:44.029807 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016de66-d667-48c7-9c43-f58f4d361ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:44.030090 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016de66-d667-48c7-9c43-f58f4d361ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:44.030360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016de66-d667-48c7-9c43-f58f4d361ad2 admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:44.030998 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1016de66-d667-48c7-9c43-f58f4d361ad2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:58:44.031297 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1016de66-d667-48c7-9c43-f58f4d361ad2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:44.032891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1016de66-d667-48c7-9c43-f58f4d361ad2 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:44.043614 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-2854b50e-69a0-4f9d-915a-ef5e95c48af6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f009e72b63144aeeac0d410685c70f14', 'request_id': 'req-2854b50e-69a0-4f9d-915a-ef5e95c48af6'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:44.045183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1150/2282] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:58:44.054853 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016de66-d667-48c7-9c43-f58f4d361ad2 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:44.065463 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016de66-d667-48c7-9c43-f58f4d361ad2 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:44.077587 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:44.120959 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8600e-e722-4b42-b3c7-2a8ccbf334c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:44.121560 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8600e-e722-4b42-b3c7-2a8ccbf334c9 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:44.121836 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8600e-e722-4b42-b3c7-2a8ccbf334c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:44.122105 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be8600e-e722-4b42-b3c7-2a8ccbf334c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:44.122727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8600e-e722-4b42-b3c7-2a8ccbf334c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:44.124641 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be8600e-e722-4b42-b3c7-2a8ccbf334c9 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:44.136928 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-2854b50e-69a0-4f9d-915a-ef5e95c48af6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f009e72b63144aeeac0d410685c70f14', 'request_id': 'req-2854b50e-69a0-4f9d-915a-ef5e95c48af6'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:44.143124 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-2854b50e-69a0-4f9d-915a-ef5e95c48af6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f009e72b63144aeeac0d410685c70f14', 'request_id': 'req-2854b50e-69a0-4f9d-915a-ef5e95c48af6'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:44.155174 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-2854b50e-69a0-4f9d-915a-ef5e95c48af6 admin admin] The token cache is being invalidate because project f009e72b63144aeeac0d410685c70f14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:58:44.155458 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-2854b50e-69a0-4f9d-915a-ef5e95c48af6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:44.156742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-2854b50e-69a0-4f9d-915a-ef5e95c48af6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:44.158218 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1133/2283] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:58:43 2022] DELETE /identity/v3/projects/f009e72b63144aeeac0d410685c70f14 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:58:44.162724 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89cc48-c00e-4898-ba92-012e0836f13d None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:44.163182 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89cc48-c00e-4898-ba92-012e0836f13d None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:44.163325 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89cc48-c00e-4898-ba92-012e0836f13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:44.187853 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-4e89cc48-c00e-4898-ba92-012e0836f13d None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:44.189282 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-4e89cc48-c00e-4898-ba92-012e0836f13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:44.191795 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1151/2284] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:58:44 2022] POST /identity/v3/projects => generated 365 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:58:44.201724 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8600e-e722-4b42-b3c7-2a8ccbf334c9 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:44.203514 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-7be8600e-e722-4b42-b3c7-2a8ccbf334c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:44.233539 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:44.255924 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1134/2285] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:58:44.262230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557f53a-b13c-4185-851e-282920205560 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:44.262514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557f53a-b13c-4185-851e-282920205560 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:44.262789 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f557f53a-b13c-4185-851e-282920205560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:44.279937 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be8600e-e722-4b42-b3c7-2a8ccbf334c9 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:44.282552 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-f557f53a-b13c-4185-851e-282920205560 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:44.283771 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-f557f53a-b13c-4185-851e-282920205560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:44.285217 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47635e-5e3b-4c58-8312-0ee3965b72d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:44.285853 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47635e-5e3b-4c58-8312-0ee3965b72d1 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:44.286133 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47635e-5e3b-4c58-8312-0ee3965b72d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:44.286401 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47635e-5e3b-4c58-8312-0ee3965b72d1 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:44.287039 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47635e-5e3b-4c58-8312-0ee3965b72d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:44.288995 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47635e-5e3b-4c58-8312-0ee3965b72d1 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:44.299392 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1135/2286] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:58:44.301381 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557f53a-b13c-4185-851e-282920205560 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:44.310601 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f557f53a-b13c-4185-851e-282920205560 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:44.315510 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ed5c9-70e6-410d-9a26-de6f049c7e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:44.316139 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ed5c9-70e6-410d-9a26-de6f049c7e4f admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:44.316421 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ed5c9-70e6-410d-9a26-de6f049c7e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:44.316693 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95ed5c9-70e6-410d-9a26-de6f049c7e4f admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:44.317337 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ed5c9-70e6-410d-9a26-de6f049c7e4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:58:44.317641 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ed5c9-70e6-410d-9a26-de6f049c7e4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:44.319303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95ed5c9-70e6-410d-9a26-de6f049c7e4f admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:44.330784 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1136/2287] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:58:44.334872 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ed5c9-70e6-410d-9a26-de6f049c7e4f admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:44.345499 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95ed5c9-70e6-410d-9a26-de6f049c7e4f admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:44.350084 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd2373-8b0b-4880-bd77-eeacd892bd27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:44.351864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd2373-8b0b-4880-bd77-eeacd892bd27 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:44.351864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd2373-8b0b-4880-bd77-eeacd892bd27 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:44.351864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd2373-8b0b-4880-bd77-eeacd892bd27 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:44.352268 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd2373-8b0b-4880-bd77-eeacd892bd27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:44.354318 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd2373-8b0b-4880-bd77-eeacd892bd27 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:44.391771 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1152/2288] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:58:44 2022] POST /identity/v3/users => generated 283 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:58:44.394990 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47635e-5e3b-4c58-8312-0ee3965b72d1 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:44.405970 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47635e-5e3b-4c58-8312-0ee3965b72d1 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:44.410579 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408889d-29e8-44e5-8627-1bd382f00a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:44.411172 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408889d-29e8-44e5-8627-1bd382f00a65 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:44.411465 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408889d-29e8-44e5-8627-1bd382f00a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:44.411721 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408889d-29e8-44e5-8627-1bd382f00a65 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:44.412387 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408889d-29e8-44e5-8627-1bd382f00a65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:44.415607 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408889d-29e8-44e5-8627-1bd382f00a65 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:44.424647 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1153/2289] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:58:44.428570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408889d-29e8-44e5-8627-1bd382f00a65 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:44.437854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408889d-29e8-44e5-8627-1bd382f00a65 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:44.442467 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16006259-4775-4b65-b346-6b008be8f12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:44.443021 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16006259-4775-4b65-b346-6b008be8f12a admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:44.443300 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16006259-4775-4b65-b346-6b008be8f12a admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:44.443574 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16006259-4775-4b65-b346-6b008be8f12a admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:44.444214 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16006259-4775-4b65-b346-6b008be8f12a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:44.445743 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16006259-4775-4b65-b346-6b008be8f12a admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:44.455344 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1154/2290] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:44 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:58:44.459147 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16006259-4775-4b65-b346-6b008be8f12a admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:44.475131 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1137/2291] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:58:44 2022] POST /identity/v3/projects => generated 387 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:58:44.478617 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd2373-8b0b-4880-bd77-eeacd892bd27 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:44.507406 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16006259-4775-4b65-b346-6b008be8f12a admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:44.534036 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:44.565157 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b836545c-94bc-4da6-8136-b07c3ba24a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:44.565822 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836545c-94bc-4da6-8136-b07c3ba24a02 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:44.566159 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836545c-94bc-4da6-8136-b07c3ba24a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:44.566434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836545c-94bc-4da6-8136-b07c3ba24a02 admin admin] PATH_INFO: `/v3/projects/fa113449244b4246b160847382cb1d42/users/eec69e195ccc44ad908bac698fbd2a0c/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:44.581877 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b836545c-94bc-4da6-8136-b07c3ba24a02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa113449244b4246b160847382cb1d42, user_id=eec69e195ccc44ad908bac698fbd2a0c, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:44.582904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd2373-8b0b-4880-bd77-eeacd892bd27 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:44.584960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b836545c-94bc-4da6-8136-b07c3ba24a02 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:44.588922 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89026f96-32f1-4036-aa66-ba5f64f26188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:44.590036 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89026f96-32f1-4036-aa66-ba5f64f26188 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:44.590537 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89026f96-32f1-4036-aa66-ba5f64f26188 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:44.595107 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89026f96-32f1-4036-aa66-ba5f64f26188 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:44.595107 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89026f96-32f1-4036-aa66-ba5f64f26188 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:44.595107 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89026f96-32f1-4036-aa66-ba5f64f26188 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:44.677751 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1138/2292] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:58:44 2022] POST /identity/v3/users => generated 294 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:58:44.678786 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1155/2293] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:58:44 2022] PUT /identity/v3/projects/fa113449244b4246b160847382cb1d42/users/eec69e195ccc44ad908bac698fbd2a0c/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:58:44.690953 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef65d7f-387c-4813-a738-b67c65c3600b None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:44.690953 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef65d7f-387c-4813-a738-b67c65c3600b None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:44.690953 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef65d7f-387c-4813-a738-b67c65c3600b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:44.692343 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89026f96-32f1-4036-aa66-ba5f64f26188 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:44.708783 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89026f96-32f1-4036-aa66-ba5f64f26188 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:44.722761 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-9ef65d7f-387c-4813-a738-b67c65c3600b None None] MFA Rules not processed for user `eec69e195ccc44ad908bac698fbd2a0c`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:44.725453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-9ef65d7f-387c-4813-a738-b67c65c3600b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:44.755981 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:44.773992 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9879a84-2f89-47f8-b43e-5e8718bccbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:44.773992 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9879a84-2f89-47f8-b43e-5e8718bccbc0 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:44.773992 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9879a84-2f89-47f8-b43e-5e8718bccbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:44.774962 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9879a84-2f89-47f8-b43e-5e8718bccbc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:44.775360 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9879a84-2f89-47f8-b43e-5e8718bccbc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:44.776978 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9879a84-2f89-47f8-b43e-5e8718bccbc0 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:44.786588 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1139/2294] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:44 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:58:44.790726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9879a84-2f89-47f8-b43e-5e8718bccbc0 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:44.792750 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1156/2295] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:58:44.797071 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef65d7f-387c-4813-a738-b67c65c3600b None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:44.801000 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9879a84-2f89-47f8-b43e-5e8718bccbc0 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:44.806735 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55ec94-db8b-4924-8bf4-fdfaa7afbdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:44.807373 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55ec94-db8b-4924-8bf4-fdfaa7afbdd4 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:44.807648 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55ec94-db8b-4924-8bf4-fdfaa7afbdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:44.807952 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55ec94-db8b-4924-8bf4-fdfaa7afbdd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:44.808590 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef55ec94-db8b-4924-8bf4-fdfaa7afbdd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:44.810168 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef55ec94-db8b-4924-8bf4-fdfaa7afbdd4 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:44.810297 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef65d7f-387c-4813-a738-b67c65c3600b None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:44.816077 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ce7c5-4456-4ec0-ad8c-305449389b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:44.816689 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ce7c5-4456-4ec0-ad8c-305449389b54 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:44.816972 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ce7c5-4456-4ec0-ad8c-305449389b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:44.817245 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ce7c5-4456-4ec0-ad8c-305449389b54 admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:44.818168 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93ce7c5-4456-4ec0-ad8c-305449389b54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:44.820298 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93ce7c5-4456-4ec0-ad8c-305449389b54 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:44.821540 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1140/2296] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:58:44.825526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55ec94-db8b-4924-8bf4-fdfaa7afbdd4 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:44.838314 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55ec94-db8b-4924-8bf4-fdfaa7afbdd4 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:44.843390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf6da96-3f2d-4d4d-b709-59b18ad271e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:44.844265 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6da96-3f2d-4d4d-b709-59b18ad271e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:44.844533 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6da96-3f2d-4d4d-b709-59b18ad271e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:44.844806 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf6da96-3f2d-4d4d-b709-59b18ad271e4 admin admin] PATH_INFO: `/v3/projects/6bb5864e3ee943708663e63bb03efae4/users/19318cdb99c5464cb1a9b21b3e333306/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:44.860258 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf6da96-3f2d-4d4d-b709-59b18ad271e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bb5864e3ee943708663e63bb03efae4, user_id=19318cdb99c5464cb1a9b21b3e333306, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:44.862965 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf6da96-3f2d-4d4d-b709-59b18ad271e4 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:44.904111 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1157/2297] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:58:44 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:58:44.908023 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ce7c5-4456-4ec0-ad8c-305449389b54 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:44.926485 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1141/2298] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:58:44 2022] PUT /identity/v3/projects/6bb5864e3ee943708663e63bb03efae4/users/19318cdb99c5464cb1a9b21b3e333306/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:58:44.938003 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68b0bc-a08a-44c9-8609-4a98d9f96388 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:44.938607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68b0bc-a08a-44c9-8609-4a98d9f96388 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:44.938890 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68b0bc-a08a-44c9-8609-4a98d9f96388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:44.968764 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-2b68b0bc-a08a-44c9-8609-4a98d9f96388 None None] MFA Rules not processed for user `19318cdb99c5464cb1a9b21b3e333306`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:44.969968 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-2b68b0bc-a08a-44c9-8609-4a98d9f96388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:44.985982 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ce7c5-4456-4ec0-ad8c-305449389b54 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:45.005956 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:45.036099 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1142/2299] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:58:45.040584 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68b0bc-a08a-44c9-8609-4a98d9f96388 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:45.042068 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3105d33-1560-40c9-b1f5-9c1e8ecd5111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:45.042679 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3105d33-1560-40c9-b1f5-9c1e8ecd5111 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:45.042965 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3105d33-1560-40c9-b1f5-9c1e8ecd5111 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:45.043239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3105d33-1560-40c9-b1f5-9c1e8ecd5111 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:45.043908 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3105d33-1560-40c9-b1f5-9c1e8ecd5111 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:45.047011 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3105d33-1560-40c9-b1f5-9c1e8ecd5111 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:45.050788 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68b0bc-a08a-44c9-8609-4a98d9f96388 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:45.055316 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096d8f4-f83e-4fd1-be34-a117881c0d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:45.056430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096d8f4-f83e-4fd1-be34-a117881c0d11 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:45.056709 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096d8f4-f83e-4fd1-be34-a117881c0d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:45.056978 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096d8f4-f83e-4fd1-be34-a117881c0d11 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:45.057602 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096d8f4-f83e-4fd1-be34-a117881c0d11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:45.059470 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096d8f4-f83e-4fd1-be34-a117881c0d11 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:45.114528 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1158/2300] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:58:44 2022] POST /identity/v3/users => generated 289 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:58:45.118721 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3105d33-1560-40c9-b1f5-9c1e8ecd5111 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:45.127914 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3105d33-1560-40c9-b1f5-9c1e8ecd5111 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:45.133417 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047337c3-120f-44f5-a055-f943b09ad0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:45.135731 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047337c3-120f-44f5-a055-f943b09ad0ae admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:45.135731 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047337c3-120f-44f5-a055-f943b09ad0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:45.135731 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047337c3-120f-44f5-a055-f943b09ad0ae admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:45.135731 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047337c3-120f-44f5-a055-f943b09ad0ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:45.137398 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047337c3-120f-44f5-a055-f943b09ad0ae admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:45.142079 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1143/2301] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:58:45 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:58:45.143283 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096d8f4-f83e-4fd1-be34-a117881c0d11 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:45.147536 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1159/2302] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:58:45.151516 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047337c3-120f-44f5-a055-f943b09ad0ae admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:45.164891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047337c3-120f-44f5-a055-f943b09ad0ae admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:45.205716 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096d8f4-f83e-4fd1-be34-a117881c0d11 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:45.211070 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc239b45-7c10-4d6a-84a4-3ed7a7b0c4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:45.211448 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc239b45-7c10-4d6a-84a4-3ed7a7b0c4e4 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:45.211774 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc239b45-7c10-4d6a-84a4-3ed7a7b0c4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:45.212058 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc239b45-7c10-4d6a-84a4-3ed7a7b0c4e4 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:45.212924 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc239b45-7c10-4d6a-84a4-3ed7a7b0c4e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:45.214817 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc239b45-7c10-4d6a-84a4-3ed7a7b0c4e4 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:45.224006 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b70cd6-ecf1-4a01-b3d8-ac5b1aefdb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:45.224653 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b70cd6-ecf1-4a01-b3d8-ac5b1aefdb4b admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:45.224934 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b70cd6-ecf1-4a01-b3d8-ac5b1aefdb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:45.225208 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b70cd6-ecf1-4a01-b3d8-ac5b1aefdb4b admin admin] PATH_INFO: `/v3/projects/4c531ddcd70c4ce0b67405120b400312/users/3f363feb0e99474195a373d00653dad4/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:45.239863 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b70cd6-ecf1-4a01-b3d8-ac5b1aefdb4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c531ddcd70c4ce0b67405120b400312, user_id=3f363feb0e99474195a373d00653dad4, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:45.242565 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b70cd6-ecf1-4a01-b3d8-ac5b1aefdb4b admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:45.309712 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:45.314845 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1144/2303] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:58:45 2022] POST /identity/v3/users => generated 299 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:58:45.318757 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc239b45-7c10-4d6a-84a4-3ed7a7b0c4e4 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:45.327985 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc239b45-7c10-4d6a-84a4-3ed7a7b0c4e4 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:45.332514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf3e11-9232-4d42-b519-b4e1166fa0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:45.333077 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf3e11-9232-4d42-b519-b4e1166fa0dc admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:45.333362 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf3e11-9232-4d42-b519-b4e1166fa0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:45.333626 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf3e11-9232-4d42-b519-b4e1166fa0dc admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:45.334253 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bf3e11-9232-4d42-b519-b4e1166fa0dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:45.335829 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bf3e11-9232-4d42-b519-b4e1166fa0dc admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:45.339709 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1160/2304] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:58:45 2022] PUT /identity/v3/projects/4c531ddcd70c4ce0b67405120b400312/users/3f363feb0e99474195a373d00653dad4/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:45.345815 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1145/2305] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:58:45.347431 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b70cd6-ecf1-4a01-b3d8-ac5b1aefdb4b admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:45.349400 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf3e11-9232-4d42-b519-b4e1166fa0dc admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:45.359232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf3e11-9232-4d42-b519-b4e1166fa0dc admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:45.412265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b70cd6-ecf1-4a01-b3d8-ac5b1aefdb4b admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:45.417171 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb245bf-9043-466d-a897-4680f84b0d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:45.417518 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be4c11-f938-4a85-b5a5-cda408628c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:45.417831 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb245bf-9043-466d-a897-4680f84b0d6d admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:45.418114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb245bf-9043-466d-a897-4680f84b0d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:45.418212 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be4c11-f938-4a85-b5a5-cda408628c4a admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:45.418396 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb245bf-9043-466d-a897-4680f84b0d6d admin admin] PATH_INFO: `/v3/projects/4b2bc48da6bd4fdfb192084058b5a6e5/users/43e48e8702f04dba9de53d9193ad90b7/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:45.418549 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be4c11-f938-4a85-b5a5-cda408628c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:45.418650 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be4c11-f938-4a85-b5a5-cda408628c4a admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:45.419294 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be4c11-f938-4a85-b5a5-cda408628c4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:45.420923 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0be4c11-f938-4a85-b5a5-cda408628c4a admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:45.431708 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1161/2306] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:58:45.434570 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb245bf-9043-466d-a897-4680f84b0d6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b2bc48da6bd4fdfb192084058b5a6e5, user_id=43e48e8702f04dba9de53d9193ad90b7, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:45.436634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be4c11-f938-4a85-b5a5-cda408628c4a admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:45.437398 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb245bf-9043-466d-a897-4680f84b0d6d admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:45.446870 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0be4c11-f938-4a85-b5a5-cda408628c4a admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:45.451637 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a26c816-227a-477c-8987-9949584a8e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:45.452519 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26c816-227a-477c-8987-9949584a8e55 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:45.452797 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26c816-227a-477c-8987-9949584a8e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:45.453106 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a26c816-227a-477c-8987-9949584a8e55 admin admin] PATH_INFO: `/v3/projects/4c531ddcd70c4ce0b67405120b400312/users/3f363feb0e99474195a373d00653dad4/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:45.455143 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26c816-227a-477c-8987-9949584a8e55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c531ddcd70c4ce0b67405120b400312, user_id=3f363feb0e99474195a373d00653dad4, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:45.457971 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a26c816-227a-477c-8987-9949584a8e55 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:45.468401 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1162/2307] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:58:45 2022] PUT /identity/v3/projects/4c531ddcd70c4ce0b67405120b400312/users/3f363feb0e99474195a373d00653dad4/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:45.479422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848dd9aa-8900-41ed-b3d8-e68efdbda4ad None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:45.479706 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848dd9aa-8900-41ed-b3d8-e68efdbda4ad None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:45.483860 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848dd9aa-8900-41ed-b3d8-e68efdbda4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:45.500083 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:45.508782 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-848dd9aa-8900-41ed-b3d8-e68efdbda4ad None None] MFA Rules not processed for user `3f363feb0e99474195a373d00653dad4`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:45.509041 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1146/2308] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:58:45 2022] PUT /identity/v3/projects/4b2bc48da6bd4fdfb192084058b5a6e5/users/43e48e8702f04dba9de53d9193ad90b7/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:58:45.510024 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-848dd9aa-8900-41ed-b3d8-e68efdbda4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:45.513034 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb245bf-9043-466d-a897-4680f84b0d6d admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:45.575258 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb245bf-9043-466d-a897-4680f84b0d6d admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:45.579011 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1163/2309] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:58:45.579984 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe20c3a-d2b7-49a9-b5db-15b2a72717ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:45.580612 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe20c3a-d2b7-49a9-b5db-15b2a72717ca admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:45.580903 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe20c3a-d2b7-49a9-b5db-15b2a72717ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:45.581168 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe20c3a-d2b7-49a9-b5db-15b2a72717ca admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:45.581810 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe20c3a-d2b7-49a9-b5db-15b2a72717ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:45.583568 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe20c3a-d2b7-49a9-b5db-15b2a72717ca admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:45.590987 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848dd9aa-8900-41ed-b3d8-e68efdbda4ad None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:45.592618 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-848dd9aa-8900-41ed-b3d8-e68efdbda4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:45.594488 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1147/2310] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:58:45.598331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe20c3a-d2b7-49a9-b5db-15b2a72717ca admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:45.608629 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe20c3a-d2b7-49a9-b5db-15b2a72717ca admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:45.613201 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19e463c-6dfc-42a6-b627-3ca9f8014569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:45.613839 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e463c-6dfc-42a6-b627-3ca9f8014569 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:45.614119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e463c-6dfc-42a6-b627-3ca9f8014569 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:45.614389 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19e463c-6dfc-42a6-b627-3ca9f8014569 admin admin] PATH_INFO: `/v3/projects/4b2bc48da6bd4fdfb192084058b5a6e5/users/43e48e8702f04dba9de53d9193ad90b7/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:45.616316 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e463c-6dfc-42a6-b627-3ca9f8014569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b2bc48da6bd4fdfb192084058b5a6e5, user_id=43e48e8702f04dba9de53d9193ad90b7, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:45.619530 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19e463c-6dfc-42a6-b627-3ca9f8014569 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:45.629960 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1148/2311] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:58:45 2022] PUT /identity/v3/projects/4b2bc48da6bd4fdfb192084058b5a6e5/users/43e48e8702f04dba9de53d9193ad90b7/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:58:45.639723 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab31ba-9dd3-4b57-b824-2231b5e00af8 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:45.640049 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab31ba-9dd3-4b57-b824-2231b5e00af8 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:45.640324 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab31ba-9dd3-4b57-b824-2231b5e00af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:45.666697 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848dd9aa-8900-41ed-b3d8-e68efdbda4ad None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:45.669398 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-6aab31ba-9dd3-4b57-b824-2231b5e00af8 None None] MFA Rules not processed for user `43e48e8702f04dba9de53d9193ad90b7`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:45.670605 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-6aab31ba-9dd3-4b57-b824-2231b5e00af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:45.716250 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe86830-9f51-4422-bdd3-1a6e5ac63b71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:45.716859 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe86830-9f51-4422-bdd3-1a6e5ac63b71 service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:45.717137 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe86830-9f51-4422-bdd3-1a6e5ac63b71 service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:45.717407 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe86830-9f51-4422-bdd3-1a6e5ac63b71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:45.717809 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:45.742124 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1149/2312] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:58:45.753813 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab31ba-9dd3-4b57-b824-2231b5e00af8 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:45.758321 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-6aab31ba-9dd3-4b57-b824-2231b5e00af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:45.765514 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:45.768156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab31ba-9dd3-4b57-b824-2231b5e00af8 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:45.771810 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e716e-8c47-4606-b138-be6a2cb0b7f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:45.772439 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e716e-8c47-4606-b138-be6a2cb0b7f5 service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:45.772717 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e716e-8c47-4606-b138-be6a2cb0b7f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:45.772988 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e716e-8c47-4606-b138-be6a2cb0b7f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:45.779572 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe86830-9f51-4422-bdd3-1a6e5ac63b71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:45.781036 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe86830-9f51-4422-bdd3-1a6e5ac63b71 service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:45.783683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e716e-8c47-4606-b138-be6a2cb0b7f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:45.785543 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e716e-8c47-4606-b138-be6a2cb0b7f5 service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:45.791153 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1164/2313] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:58:45 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:58:45.796002 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1150/2314] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:58:45 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:58:46.927981 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe86830-9f51-4422-bdd3-1a6e5ac63b71 service cinder] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:46.992445 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe86830-9f51-4422-bdd3-1a6e5ac63b71 service cinder] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:46.996995 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb9663-99a4-457f-ae0b-cd38e95deeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:46.997645 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb9663-99a4-457f-ae0b-cd38e95deeba admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:46.997971 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb9663-99a4-457f-ae0b-cd38e95deeba admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:46.998250 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb9663-99a4-457f-ae0b-cd38e95deeba admin admin] PATH_INFO: `/v3/users/eec69e195ccc44ad908bac698fbd2a0c` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:46.999370 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bb9663-99a4-457f-ae0b-cd38e95deeba admin admin] RBAC: Authorizing `identity:delete_user(user_id=eec69e195ccc44ad908bac698fbd2a0c)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:47.001331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bb9663-99a4-457f-ae0b-cd38e95deeba admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:47.186748 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-50bb9663-99a4-457f-ae0b-cd38e95deeba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eec69e195ccc44ad908bac698fbd2a0c'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:47.187464 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-50bb9663-99a4-457f-ae0b-cd38e95deeba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eec69e195ccc44ad908bac698fbd2a0c'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:47.195683 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e716e-8c47-4606-b138-be6a2cb0b7f5 service cinder] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:47.206363 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e716e-8c47-4606-b138-be6a2cb0b7f5 service cinder] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:47.210865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf1a3d-7eb2-4016-b6c0-78feb8523d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:47.211478 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf1a3d-7eb2-4016-b6c0-78feb8523d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:47.211827 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf1a3d-7eb2-4016-b6c0-78feb8523d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:47.212093 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf1a3d-7eb2-4016-b6c0-78feb8523d75 admin admin] PATH_INFO: `/v3/users/19318cdb99c5464cb1a9b21b3e333306` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:47.220373 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf1a3d-7eb2-4016-b6c0-78feb8523d75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19318cdb99c5464cb1a9b21b3e333306)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:47.220373 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf1a3d-7eb2-4016-b6c0-78feb8523d75 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:47.262491 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-50bb9663-99a4-457f-ae0b-cd38e95deeba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eec69e195ccc44ad908bac698fbd2a0c'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:47.275136 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-50bb9663-99a4-457f-ae0b-cd38e95deeba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eec69e195ccc44ad908bac698fbd2a0c'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:47.275797 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-50bb9663-99a4-457f-ae0b-cd38e95deeba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eec69e195ccc44ad908bac698fbd2a0c'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:47.288141 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1165/2315] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:58:46 2022] DELETE /identity/v3/users/eec69e195ccc44ad908bac698fbd2a0c => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:47.292331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9865468-76ca-4866-bdc2-1e8f07c882d3 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:47.294997 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9865468-76ca-4866-bdc2-1e8f07c882d3 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:47.294997 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9865468-76ca-4866-bdc2-1e8f07c882d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:47.321851 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-a9865468-76ca-4866-bdc2-1e8f07c882d3 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:47.323692 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-a9865468-76ca-4866-bdc2-1e8f07c882d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:47.371833 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:47.381043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1166/2316] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:58:47.392695 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9865468-76ca-4866-bdc2-1e8f07c882d3 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:47.394544 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-a9865468-76ca-4866-bdc2-1e8f07c882d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:47.398137 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-6eaf1a3d-7eb2-4016-b6c0-78feb8523d75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19318cdb99c5464cb1a9b21b3e333306'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:47.451119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-6eaf1a3d-7eb2-4016-b6c0-78feb8523d75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19318cdb99c5464cb1a9b21b3e333306'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:47.452653 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9865468-76ca-4866-bdc2-1e8f07c882d3 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:47.456322 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854643ef-a97c-4bb6-9807-40b47128f0da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:47.456928 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854643ef-a97c-4bb6-9807-40b47128f0da service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:47.457210 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854643ef-a97c-4bb6-9807-40b47128f0da service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:47.457481 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854643ef-a97c-4bb6-9807-40b47128f0da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:47.464201 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-6eaf1a3d-7eb2-4016-b6c0-78feb8523d75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19318cdb99c5464cb1a9b21b3e333306'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:47.464858 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-6eaf1a3d-7eb2-4016-b6c0-78feb8523d75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19318cdb99c5464cb1a9b21b3e333306'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:47.465567 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-6eaf1a3d-7eb2-4016-b6c0-78feb8523d75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19318cdb99c5464cb1a9b21b3e333306'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:47.466898 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854643ef-a97c-4bb6-9807-40b47128f0da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:47.468346 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854643ef-a97c-4bb6-9807-40b47128f0da service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:47.478425 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1151/2317] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:58:47 2022] DELETE /identity/v3/users/19318cdb99c5464cb1a9b21b3e333306 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:58:47.481814 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1167/2318] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:58:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:58:47.482754 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cbc49-f0cd-47f8-b8ac-3a989de10bc3 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:47.483044 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cbc49-f0cd-47f8-b8ac-3a989de10bc3 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:47.483323 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cbc49-f0cd-47f8-b8ac-3a989de10bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:47.502891 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-438cbc49-f0cd-47f8-b8ac-3a989de10bc3 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:47.504149 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-438cbc49-f0cd-47f8-b8ac-3a989de10bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:47.564357 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1152/2319] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:58:47.574049 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854643ef-a97c-4bb6-9807-40b47128f0da service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:47.575678 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-854643ef-a97c-4bb6-9807-40b47128f0da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:47.588171 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854643ef-a97c-4bb6-9807-40b47128f0da service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:47.639513 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93624208-57fd-4bfa-b815-f0a6cb08e171 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:47.640152 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93624208-57fd-4bfa-b815-f0a6cb08e171 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:47.640438 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93624208-57fd-4bfa-b815-f0a6cb08e171 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:47.640692 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93624208-57fd-4bfa-b815-f0a6cb08e171 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:47.650347 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93624208-57fd-4bfa-b815-f0a6cb08e171 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:47.651828 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93624208-57fd-4bfa-b815-f0a6cb08e171 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:47.661180 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1168/2320] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:58:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:58:47.967340 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438cbc49-f0cd-47f8-b8ac-3a989de10bc3 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:47.969231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-438cbc49-f0cd-47f8-b8ac-3a989de10bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:47.981195 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438cbc49-f0cd-47f8-b8ac-3a989de10bc3 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:47.985823 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300651b4-d42a-4a32-bd3c-7cfb06dac092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:47.986470 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300651b4-d42a-4a32-bd3c-7cfb06dac092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:47.986752 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300651b4-d42a-4a32-bd3c-7cfb06dac092 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:47.987109 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300651b4-d42a-4a32-bd3c-7cfb06dac092 admin admin] PATH_INFO: `/v3/projects/fa113449244b4246b160847382cb1d42` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:47.988291 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300651b4-d42a-4a32-bd3c-7cfb06dac092 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa113449244b4246b160847382cb1d42)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:47.990240 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300651b4-d42a-4a32-bd3c-7cfb06dac092 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:48.116496 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:48.126563 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-300651b4-d42a-4a32-bd3c-7cfb06dac092 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa113449244b4246b160847382cb1d42', 'request_id': 'req-300651b4-d42a-4a32-bd3c-7cfb06dac092'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:48.139309 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93624208-57fd-4bfa-b815-f0a6cb08e171 service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:48.140847 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-93624208-57fd-4bfa-b815-f0a6cb08e171 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:48.178904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93624208-57fd-4bfa-b815-f0a6cb08e171 service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:48.243952 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8822e09-6541-4ed4-83c4-22216721562f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:48.244656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8822e09-6541-4ed4-83c4-22216721562f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:48.244916 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8822e09-6541-4ed4-83c4-22216721562f admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:48.245188 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8822e09-6541-4ed4-83c4-22216721562f admin admin] PATH_INFO: `/v3/projects/6bb5864e3ee943708663e63bb03efae4` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:48.246316 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8822e09-6541-4ed4-83c4-22216721562f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bb5864e3ee943708663e63bb03efae4)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:48.248274 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8822e09-6541-4ed4-83c4-22216721562f admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:48.274285 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-300651b4-d42a-4a32-bd3c-7cfb06dac092 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa113449244b4246b160847382cb1d42', 'request_id': 'req-300651b4-d42a-4a32-bd3c-7cfb06dac092'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:48.280422 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-300651b4-d42a-4a32-bd3c-7cfb06dac092 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa113449244b4246b160847382cb1d42', 'request_id': 'req-300651b4-d42a-4a32-bd3c-7cfb06dac092'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:48.293447 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-300651b4-d42a-4a32-bd3c-7cfb06dac092 admin admin] The token cache is being invalidate because project fa113449244b4246b160847382cb1d42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:58:48.293810 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-300651b4-d42a-4a32-bd3c-7cfb06dac092 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:48.294524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-300651b4-d42a-4a32-bd3c-7cfb06dac092 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:48.296171 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1153/2321] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:58:47 2022] DELETE /identity/v3/projects/fa113449244b4246b160847382cb1d42 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:48.300043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300651b4-d42a-4a32-bd3c-7cfb06dac092 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:48.301444 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-300651b4-d42a-4a32-bd3c-7cfb06dac092 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:48.314881 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300651b4-d42a-4a32-bd3c-7cfb06dac092 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:48.319772 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad70a4-4ae5-4931-a1d3-6b4c38e06f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:48.320558 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad70a4-4ae5-4931-a1d3-6b4c38e06f93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:48.320934 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad70a4-4ae5-4931-a1d3-6b4c38e06f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:48.321283 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad70a4-4ae5-4931-a1d3-6b4c38e06f93 admin admin] PATH_INFO: `/v3/users/3f363feb0e99474195a373d00653dad4` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:48.322884 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad70a4-4ae5-4931-a1d3-6b4c38e06f93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f363feb0e99474195a373d00653dad4)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:48.325397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad70a4-4ae5-4931-a1d3-6b4c38e06f93 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:48.347930 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:48.350929 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-c8822e09-6541-4ed4-83c4-22216721562f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bb5864e3ee943708663e63bb03efae4', 'request_id': 'req-c8822e09-6541-4ed4-83c4-22216721562f'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:48.417134 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-c8822e09-6541-4ed4-83c4-22216721562f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bb5864e3ee943708663e63bb03efae4', 'request_id': 'req-c8822e09-6541-4ed4-83c4-22216721562f'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:48.423531 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-c8822e09-6541-4ed4-83c4-22216721562f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bb5864e3ee943708663e63bb03efae4', 'request_id': 'req-c8822e09-6541-4ed4-83c4-22216721562f'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:48.435612 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-c8822e09-6541-4ed4-83c4-22216721562f admin admin] The token cache is being invalidate because project 6bb5864e3ee943708663e63bb03efae4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:58:48.435921 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-c8822e09-6541-4ed4-83c4-22216721562f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:48.436585 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-c8822e09-6541-4ed4-83c4-22216721562f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:48.438093 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1169/2322] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:58:48 2022] DELETE /identity/v3/projects/6bb5864e3ee943708663e63bb03efae4 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:48.441878 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8822e09-6541-4ed4-83c4-22216721562f admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:48.443156 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-c8822e09-6541-4ed4-83c4-22216721562f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:48.505523 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8822e09-6541-4ed4-83c4-22216721562f admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:48.510568 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7dd217-bb52-4093-a95f-e8fb962195d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:48.511181 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7dd217-bb52-4093-a95f-e8fb962195d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:48.511466 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7dd217-bb52-4093-a95f-e8fb962195d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:48.511943 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7dd217-bb52-4093-a95f-e8fb962195d0 admin admin] PATH_INFO: `/v3/users/43e48e8702f04dba9de53d9193ad90b7` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:48.513101 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7dd217-bb52-4093-a95f-e8fb962195d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43e48e8702f04dba9de53d9193ad90b7)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:48.515044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7dd217-bb52-4093-a95f-e8fb962195d0 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:48.524752 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-5cad70a4-4ae5-4931-a1d3-6b4c38e06f93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f363feb0e99474195a373d00653dad4'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:48.613410 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-5cad70a4-4ae5-4931-a1d3-6b4c38e06f93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f363feb0e99474195a373d00653dad4'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:48.628390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-5cad70a4-4ae5-4931-a1d3-6b4c38e06f93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f363feb0e99474195a373d00653dad4'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:48.630364 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-5cad70a4-4ae5-4931-a1d3-6b4c38e06f93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f363feb0e99474195a373d00653dad4'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:48.630364 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-5cad70a4-4ae5-4931-a1d3-6b4c38e06f93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f363feb0e99474195a373d00653dad4'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:48.642162 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1154/2323] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:58:48 2022] DELETE /identity/v3/users/3f363feb0e99474195a373d00653dad4 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:58:48.714009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3a7dd217-bb52-4093-a95f-e8fb962195d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43e48e8702f04dba9de53d9193ad90b7'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:48.714837 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3a7dd217-bb52-4093-a95f-e8fb962195d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43e48e8702f04dba9de53d9193ad90b7'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:48.767444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3a7dd217-bb52-4093-a95f-e8fb962195d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43e48e8702f04dba9de53d9193ad90b7'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:48.780810 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3a7dd217-bb52-4093-a95f-e8fb962195d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43e48e8702f04dba9de53d9193ad90b7'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:48.781468 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3a7dd217-bb52-4093-a95f-e8fb962195d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43e48e8702f04dba9de53d9193ad90b7'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:48.794672 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1170/2324] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:58:48 2022] DELETE /identity/v3/users/43e48e8702f04dba9de53d9193ad90b7 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:58:49.104065 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad70a4-4ae5-4931-a1d3-6b4c38e06f93 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:49.105626 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-5cad70a4-4ae5-4931-a1d3-6b4c38e06f93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:49.185326 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad70a4-4ae5-4931-a1d3-6b4c38e06f93 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:49.226164 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:49.242381 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f6042d-2e23-4a94-98fa-399944b46425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:49.243024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6042d-2e23-4a94-98fa-399944b46425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:49.243472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6042d-2e23-4a94-98fa-399944b46425 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:49.248585 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f6042d-2e23-4a94-98fa-399944b46425 admin admin] PATH_INFO: `/v3/projects/4c531ddcd70c4ce0b67405120b400312` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:49.248585 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6042d-2e23-4a94-98fa-399944b46425 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c531ddcd70c4ce0b67405120b400312)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:49.248585 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f6042d-2e23-4a94-98fa-399944b46425 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:49.250573 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7dd217-bb52-4093-a95f-e8fb962195d0 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:49.252587 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-3a7dd217-bb52-4093-a95f-e8fb962195d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:49.278245 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7dd217-bb52-4093-a95f-e8fb962195d0 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:49.296924 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e6c1b6-13d1-4780-91f7-aa9695a84897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:49.297586 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6c1b6-13d1-4780-91f7-aa9695a84897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:49.298144 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6c1b6-13d1-4780-91f7-aa9695a84897 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:49.298144 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6c1b6-13d1-4780-91f7-aa9695a84897 admin admin] PATH_INFO: `/v3/projects/4b2bc48da6bd4fdfb192084058b5a6e5` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:49.299293 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e6c1b6-13d1-4780-91f7-aa9695a84897 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b2bc48da6bd4fdfb192084058b5a6e5)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:49.301389 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e6c1b6-13d1-4780-91f7-aa9695a84897 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:49.354940 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-44f6042d-2e23-4a94-98fa-399944b46425 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c531ddcd70c4ce0b67405120b400312', 'request_id': 'req-44f6042d-2e23-4a94-98fa-399944b46425'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:49.410847 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:49.412266 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-32e6c1b6-13d1-4780-91f7-aa9695a84897 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b2bc48da6bd4fdfb192084058b5a6e5', 'request_id': 'req-32e6c1b6-13d1-4780-91f7-aa9695a84897'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:49.422647 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-44f6042d-2e23-4a94-98fa-399944b46425 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c531ddcd70c4ce0b67405120b400312', 'request_id': 'req-44f6042d-2e23-4a94-98fa-399944b46425'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:49.429264 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-44f6042d-2e23-4a94-98fa-399944b46425 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c531ddcd70c4ce0b67405120b400312', 'request_id': 'req-44f6042d-2e23-4a94-98fa-399944b46425'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:49.441184 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-44f6042d-2e23-4a94-98fa-399944b46425 admin admin] The token cache is being invalidate because project 4c531ddcd70c4ce0b67405120b400312 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:58:49.441477 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-44f6042d-2e23-4a94-98fa-399944b46425 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:49.442148 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-44f6042d-2e23-4a94-98fa-399944b46425 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:49.443606 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1155/2325] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:58:49 2022] DELETE /identity/v3/projects/4c531ddcd70c4ce0b67405120b400312 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:49.449238 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182c686-324a-467a-af7c-9fe88a039de2 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:49.449547 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182c686-324a-467a-af7c-9fe88a039de2 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:49.457546 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182c686-324a-467a-af7c-9fe88a039de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:49.499812 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-6182c686-324a-467a-af7c-9fe88a039de2 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:49.501058 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-6182c686-324a-467a-af7c-9fe88a039de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:49.526075 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-32e6c1b6-13d1-4780-91f7-aa9695a84897 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b2bc48da6bd4fdfb192084058b5a6e5', 'request_id': 'req-32e6c1b6-13d1-4780-91f7-aa9695a84897'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:49.532331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-32e6c1b6-13d1-4780-91f7-aa9695a84897 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b2bc48da6bd4fdfb192084058b5a6e5', 'request_id': 'req-32e6c1b6-13d1-4780-91f7-aa9695a84897'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:49.544137 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-32e6c1b6-13d1-4780-91f7-aa9695a84897 admin admin] The token cache is being invalidate because project 4b2bc48da6bd4fdfb192084058b5a6e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:58:49.544573 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-32e6c1b6-13d1-4780-91f7-aa9695a84897 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:49.545234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-32e6c1b6-13d1-4780-91f7-aa9695a84897 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:49.546773 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1171/2326] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:58:49 2022] DELETE /identity/v3/projects/4b2bc48da6bd4fdfb192084058b5a6e5 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:58:49.552554 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf130672-7834-4996-afdc-d4fcad62c8e0 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:49.552848 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf130672-7834-4996-afdc-d4fcad62c8e0 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:49.553105 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf130672-7834-4996-afdc-d4fcad62c8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:49.568520 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1156/2327] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:58:49.576997 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-bf130672-7834-4996-afdc-d4fcad62c8e0 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:49.578212 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f456b-f7c7-437f-bd89-7cad7971e324 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:49.578341 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-bf130672-7834-4996-afdc-d4fcad62c8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:49.578525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f456b-f7c7-437f-bd89-7cad7971e324 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:49.578801 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f456b-f7c7-437f-bd89-7cad7971e324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:49.592028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1172/2328] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:58:49.594241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25449569-0809-491c-98d8-9e0ec356bd23 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:49.594524 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25449569-0809-491c-98d8-9e0ec356bd23 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:49.594796 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25449569-0809-491c-98d8-9e0ec356bd23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:49.603508 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-2f2f456b-f7c7-437f-bd89-7cad7971e324 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:49.604837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-2f2f456b-f7c7-437f-bd89-7cad7971e324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:49.613041 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1157/2329] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:58:49.614479 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-25449569-0809-491c-98d8-9e0ec356bd23 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:49.616047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-25449569-0809-491c-98d8-9e0ec356bd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:49.617563 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2f456b-f7c7-437f-bd89-7cad7971e324 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:49.624953 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1173/2330] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:58:49.628161 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2f456b-f7c7-437f-bd89-7cad7971e324 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:49.629626 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25449569-0809-491c-98d8-9e0ec356bd23 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:49.634664 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0038ff-b7a4-4de8-b84f-bded7d5e2e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:49.635462 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0038ff-b7a4-4de8-b84f-bded7d5e2e8b admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:49.635762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0038ff-b7a4-4de8-b84f-bded7d5e2e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:49.636043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0038ff-b7a4-4de8-b84f-bded7d5e2e8b admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:49.636747 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0038ff-b7a4-4de8-b84f-bded7d5e2e8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:58:49.637056 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0038ff-b7a4-4de8-b84f-bded7d5e2e8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:49.638871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0038ff-b7a4-4de8-b84f-bded7d5e2e8b admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:49.639601 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25449569-0809-491c-98d8-9e0ec356bd23 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:49.644259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87c0bd-7897-4f68-8e4a-254b7ff654bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:49.644861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87c0bd-7897-4f68-8e4a-254b7ff654bc admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:49.645135 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87c0bd-7897-4f68-8e4a-254b7ff654bc admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:49.645406 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87c0bd-7897-4f68-8e4a-254b7ff654bc admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:49.646047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd87c0bd-7897-4f68-8e4a-254b7ff654bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:58:49.646345 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd87c0bd-7897-4f68-8e4a-254b7ff654bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:49.647999 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd87c0bd-7897-4f68-8e4a-254b7ff654bc admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:49.653721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1158/2331] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:58:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:58:49.660030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0038ff-b7a4-4de8-b84f-bded7d5e2e8b admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:49.661535 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1174/2332] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:58:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:58:49.665663 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87c0bd-7897-4f68-8e4a-254b7ff654bc admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:49.670253 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0038ff-b7a4-4de8-b84f-bded7d5e2e8b admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:49.675298 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40043da5-1331-48a2-946e-507b31e2ae47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:49.675945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40043da5-1331-48a2-946e-507b31e2ae47 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:49.676241 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40043da5-1331-48a2-946e-507b31e2ae47 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:49.676613 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40043da5-1331-48a2-946e-507b31e2ae47 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:49.676831 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87c0bd-7897-4f68-8e4a-254b7ff654bc admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:49.677249 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40043da5-1331-48a2-946e-507b31e2ae47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:49.679153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40043da5-1331-48a2-946e-507b31e2ae47 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:49.682612 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17813f3-310c-4dd7-844c-76da8ba8ee23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:49.683344 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17813f3-310c-4dd7-844c-76da8ba8ee23 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:49.683686 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17813f3-310c-4dd7-844c-76da8ba8ee23 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:49.684051 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17813f3-310c-4dd7-844c-76da8ba8ee23 admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:49.686996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17813f3-310c-4dd7-844c-76da8ba8ee23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:49.687908 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17813f3-310c-4dd7-844c-76da8ba8ee23 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:49.770385 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1159/2333] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:58:49 2022] POST /identity/v3/projects => generated 363 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:58:49.771435 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40043da5-1331-48a2-946e-507b31e2ae47 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:49.792343 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:49.799126 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1175/2334] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:58:49 2022] POST /identity/v3/projects => generated 377 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:58:49.803175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17813f3-310c-4dd7-844c-76da8ba8ee23 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:49.835660 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40043da5-1331-48a2-946e-507b31e2ae47 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:49.875494 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17813f3-310c-4dd7-844c-76da8ba8ee23 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:49.883207 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0573a-83ce-45c5-826b-028594f11877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:49.884132 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0573a-83ce-45c5-826b-028594f11877 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:49.884534 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0573a-83ce-45c5-826b-028594f11877 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:49.885029 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0573a-83ce-45c5-826b-028594f11877 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:49.885448 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0573a-83ce-45c5-826b-028594f11877 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:49.887487 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0573a-83ce-45c5-826b-028594f11877 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:49.901951 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149b945-6805-425c-914f-b10371f3386a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:49.903015 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149b945-6805-425c-914f-b10371f3386a admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:49.903274 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149b945-6805-425c-914f-b10371f3386a admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:49.903592 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149b945-6805-425c-914f-b10371f3386a admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:49.904236 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5149b945-6805-425c-914f-b10371f3386a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:49.906200 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5149b945-6805-425c-914f-b10371f3386a admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:49.968621 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1176/2335] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:58:49 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:58:49.972345 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0573a-83ce-45c5-826b-028594f11877 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:49.982683 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0573a-83ce-45c5-826b-028594f11877 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:49.987744 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e10752-d024-4923-b045-235d6985ead8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:49.988475 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e10752-d024-4923-b045-235d6985ead8 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:49.988783 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e10752-d024-4923-b045-235d6985ead8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:49.989034 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e10752-d024-4923-b045-235d6985ead8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:49.989673 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e10752-d024-4923-b045-235d6985ead8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:49.991258 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e10752-d024-4923-b045-235d6985ead8 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:49.995499 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1160/2336] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:58:49 2022] POST /identity/v3/users => generated 282 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:58:49.996443 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149b945-6805-425c-914f-b10371f3386a admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:50.002427 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1177/2337] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:58:50.006044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e10752-d024-4923-b045-235d6985ead8 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:50.009565 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5149b945-6805-425c-914f-b10371f3386a admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:50.014422 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7ac08-9ffb-4241-b807-29782042f870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:50.015021 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7ac08-9ffb-4241-b807-29782042f870 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:50.015299 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7ac08-9ffb-4241-b807-29782042f870 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:50.015570 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7ac08-9ffb-4241-b807-29782042f870 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:50.016262 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7ac08-9ffb-4241-b807-29782042f870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:50.016393 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e10752-d024-4923-b045-235d6985ead8 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:50.017958 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7ac08-9ffb-4241-b807-29782042f870 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:50.021367 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357391d-f83a-4a08-9243-8769d93a2005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:50.022089 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357391d-f83a-4a08-9243-8769d93a2005 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:50.022397 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357391d-f83a-4a08-9243-8769d93a2005 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:50.022757 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357391d-f83a-4a08-9243-8769d93a2005 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:50.023461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f357391d-f83a-4a08-9243-8769d93a2005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:50.025206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f357391d-f83a-4a08-9243-8769d93a2005 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:50.028438 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1161/2338] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:58:50.032044 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7ac08-9ffb-4241-b807-29782042f870 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:50.040548 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1178/2339] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:58:50.043905 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7ac08-9ffb-4241-b807-29782042f870 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:50.044962 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357391d-f83a-4a08-9243-8769d93a2005 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:50.050626 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac05f28-b5cd-4132-9fc4-2bf3eeb2d948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:50.051447 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac05f28-b5cd-4132-9fc4-2bf3eeb2d948 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:50.051936 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac05f28-b5cd-4132-9fc4-2bf3eeb2d948 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:50.052374 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac05f28-b5cd-4132-9fc4-2bf3eeb2d948 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:50.053027 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac05f28-b5cd-4132-9fc4-2bf3eeb2d948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:50.054782 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac05f28-b5cd-4132-9fc4-2bf3eeb2d948 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:50.054910 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357391d-f83a-4a08-9243-8769d93a2005 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:50.061741 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc991065-04d7-4e01-8d14-cd601d297e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:50.066169 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc991065-04d7-4e01-8d14-cd601d297e25 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:50.066169 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc991065-04d7-4e01-8d14-cd601d297e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:50.066169 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc991065-04d7-4e01-8d14-cd601d297e25 admin admin] PATH_INFO: `/v3/projects/f613f0eb6cd14f9daadf8694b60f6aa8/users/e50692a6da234d2abaee5cdfd0cfd268/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:50.067562 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1162/2340] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:58:50.071685 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac05f28-b5cd-4132-9fc4-2bf3eeb2d948 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:50.078824 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc991065-04d7-4e01-8d14-cd601d297e25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f613f0eb6cd14f9daadf8694b60f6aa8, user_id=e50692a6da234d2abaee5cdfd0cfd268, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:50.082416 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac05f28-b5cd-4132-9fc4-2bf3eeb2d948 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:50.086055 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc991065-04d7-4e01-8d14-cd601d297e25 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:50.088729 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997567f-81ca-46cb-a364-22de9ca455ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:50.089587 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997567f-81ca-46cb-a364-22de9ca455ce admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:50.089928 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997567f-81ca-46cb-a364-22de9ca455ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:50.094814 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997567f-81ca-46cb-a364-22de9ca455ce admin admin] PATH_INFO: `/v3/projects/e0c6329adddf4cbcbfe6f87a5b94124a/users/23b2b6e2ac4a48f697226b53f7a27a78/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:50.107085 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997567f-81ca-46cb-a364-22de9ca455ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0c6329adddf4cbcbfe6f87a5b94124a, user_id=23b2b6e2ac4a48f697226b53f7a27a78, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:50.109799 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997567f-81ca-46cb-a364-22de9ca455ce admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:50.194558 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1179/2341] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:58:50 2022] PUT /identity/v3/projects/f613f0eb6cd14f9daadf8694b60f6aa8/users/e50692a6da234d2abaee5cdfd0cfd268/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:58:50.205624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044d10c-e03f-4f81-88fa-92f08d163451 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:50.205911 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044d10c-e03f-4f81-88fa-92f08d163451 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:50.206192 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044d10c-e03f-4f81-88fa-92f08d163451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:50.224422 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1163/2342] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:58:50 2022] PUT /identity/v3/projects/e0c6329adddf4cbcbfe6f87a5b94124a/users/23b2b6e2ac4a48f697226b53f7a27a78/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:58:50.235391 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcd4d8-6a77-4652-81ce-29ddab982f24 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:50.235705 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcd4d8-6a77-4652-81ce-29ddab982f24 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:50.236019 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcd4d8-6a77-4652-81ce-29ddab982f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:50.236675 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-e044d10c-e03f-4f81-88fa-92f08d163451 None None] MFA Rules not processed for user `e50692a6da234d2abaee5cdfd0cfd268`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:50.237980 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-e044d10c-e03f-4f81-88fa-92f08d163451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:50.258456 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:50.264118 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-dcfcd4d8-6a77-4652-81ce-29ddab982f24 None None] MFA Rules not processed for user `23b2b6e2ac4a48f697226b53f7a27a78`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:50.265331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-dcfcd4d8-6a77-4652-81ce-29ddab982f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:50.307545 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1180/2343] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:58:50.316165 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:50.320509 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044d10c-e03f-4f81-88fa-92f08d163451 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:50.322161 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-e044d10c-e03f-4f81-88fa-92f08d163451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:50.343314 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1164/2344] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:58:50.348329 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfcd4d8-6a77-4652-81ce-29ddab982f24 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:50.384886 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044d10c-e03f-4f81-88fa-92f08d163451 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:50.388574 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0986bc-6204-4379-bbbd-8146ddf40011 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '565a836c7f89478cad615bd6c14531a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:50.389180 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0986bc-6204-4379-bbbd-8146ddf40011 service glance] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:50.389457 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0986bc-6204-4379-bbbd-8146ddf40011 service glance] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:50.389727 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0986bc-6204-4379-bbbd-8146ddf40011 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:50.398184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0986bc-6204-4379-bbbd-8146ddf40011 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:50.399675 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0986bc-6204-4379-bbbd-8146ddf40011 service glance] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:50.408613 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1181/2345] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:58:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:58:50.411247 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfcd4d8-6a77-4652-81ce-29ddab982f24 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:50.417068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05547c3f-8e64-4e75-9e80-f87bd9acae20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:50.417682 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05547c3f-8e64-4e75-9e80-f87bd9acae20 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:50.418019 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05547c3f-8e64-4e75-9e80-f87bd9acae20 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:50.418261 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05547c3f-8e64-4e75-9e80-f87bd9acae20 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:50.420933 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05547c3f-8e64-4e75-9e80-f87bd9acae20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:50.420933 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05547c3f-8e64-4e75-9e80-f87bd9acae20 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:50.533711 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1165/2346] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:58:50 2022] POST /identity/v3/projects => generated 363 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:58:50.537730 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0986bc-6204-4379-bbbd-8146ddf40011 service glance] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:50.549732 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0986bc-6204-4379-bbbd-8146ddf40011 service glance] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:50.609703 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d201d-14ce-4cd8-b53d-ca95e44ad68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:50.610368 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d201d-14ce-4cd8-b53d-ca95e44ad68f admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:50.610913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d201d-14ce-4cd8-b53d-ca95e44ad68f admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:50.611447 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535d201d-14ce-4cd8-b53d-ca95e44ad68f admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:50.612889 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535d201d-14ce-4cd8-b53d-ca95e44ad68f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:50.614803 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535d201d-14ce-4cd8-b53d-ca95e44ad68f admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:50.714825 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1182/2347] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:58:50 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:58:50.718770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05547c3f-8e64-4e75-9e80-f87bd9acae20 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:50.729511 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05547c3f-8e64-4e75-9e80-f87bd9acae20 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:50.734462 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2ffcc-6506-4597-b9ba-16eb1a6d55e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:50.735054 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2ffcc-6506-4597-b9ba-16eb1a6d55e1 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:50.736416 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2ffcc-6506-4597-b9ba-16eb1a6d55e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:50.736416 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f2ffcc-6506-4597-b9ba-16eb1a6d55e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:50.736416 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2ffcc-6506-4597-b9ba-16eb1a6d55e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:50.737810 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f2ffcc-6506-4597-b9ba-16eb1a6d55e1 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:50.748142 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1166/2348] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:58:50.752396 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d201d-14ce-4cd8-b53d-ca95e44ad68f admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:50.762631 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535d201d-14ce-4cd8-b53d-ca95e44ad68f admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:50.767325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f838be-55a5-40e2-8d9b-713ffcd1b2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:50.768000 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f838be-55a5-40e2-8d9b-713ffcd1b2e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:50.768296 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f838be-55a5-40e2-8d9b-713ffcd1b2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:50.768570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f838be-55a5-40e2-8d9b-713ffcd1b2e4 admin admin] PATH_INFO: `/v3/projects/faba5883c496433db39d4f723401082a/users/6f519e36ff6148cfaf102a12917bc689/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:50.783036 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f838be-55a5-40e2-8d9b-713ffcd1b2e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faba5883c496433db39d4f723401082a, user_id=6f519e36ff6148cfaf102a12917bc689, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:50.785794 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f838be-55a5-40e2-8d9b-713ffcd1b2e4 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:50.888134 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1183/2349] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:58:50 2022] PUT /identity/v3/projects/faba5883c496433db39d4f723401082a/users/6f519e36ff6148cfaf102a12917bc689/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:50.892015 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2ffcc-6506-4597-b9ba-16eb1a6d55e1 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:50.902727 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f2ffcc-6506-4597-b9ba-16eb1a6d55e1 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:50.957971 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff611b60-26ef-4aa5-ae03-88917abec67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:50.958667 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff611b60-26ef-4aa5-ae03-88917abec67d admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:50.958964 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff611b60-26ef-4aa5-ae03-88917abec67d admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:50.959216 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff611b60-26ef-4aa5-ae03-88917abec67d admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:50.959890 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff611b60-26ef-4aa5-ae03-88917abec67d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:50.961465 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff611b60-26ef-4aa5-ae03-88917abec67d admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:50.972159 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1167/2350] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:50 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:58:50.976459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f838be-55a5-40e2-8d9b-713ffcd1b2e4 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:50.987793 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f838be-55a5-40e2-8d9b-713ffcd1b2e4 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:50.992511 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a47868-142d-414d-9ef5-5eec2f5bed38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:50.993204 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a47868-142d-414d-9ef5-5eec2f5bed38 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:50.993512 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a47868-142d-414d-9ef5-5eec2f5bed38 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:50.993790 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a47868-142d-414d-9ef5-5eec2f5bed38 admin admin] PATH_INFO: `/v3/projects/faba5883c496433db39d4f723401082a/users/6f519e36ff6148cfaf102a12917bc689/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:50.995907 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a47868-142d-414d-9ef5-5eec2f5bed38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faba5883c496433db39d4f723401082a, user_id=6f519e36ff6148cfaf102a12917bc689, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:50.998507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a47868-142d-414d-9ef5-5eec2f5bed38 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:51.008840 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1184/2351] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:58:50 2022] PUT /identity/v3/projects/faba5883c496433db39d4f723401082a/users/6f519e36ff6148cfaf102a12917bc689/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:51.019164 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2fd30-ac8b-4079-8108-cd0fc0c4a88d None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:51.019454 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2fd30-ac8b-4079-8108-cd0fc0c4a88d None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:51.019731 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2fd30-ac8b-4079-8108-cd0fc0c4a88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:51.048248 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-c8f2fd30-ac8b-4079-8108-cd0fc0c4a88d None None] MFA Rules not processed for user `6f519e36ff6148cfaf102a12917bc689`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:51.049447 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c8f2fd30-ac8b-4079-8108-cd0fc0c4a88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:51.097090 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:51.120990 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1168/2352] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:51 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:58:51.132193 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a47868-142d-414d-9ef5-5eec2f5bed38 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:51.133800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-12a47868-142d-414d-9ef5-5eec2f5bed38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:51.153249 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:51.191214 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a47868-142d-414d-9ef5-5eec2f5bed38 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:51.195041 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5498f350-bbd1-43da-b16c-289f2705a04a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:51.195691 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498f350-bbd1-43da-b16c-289f2705a04a service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:51.196035 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498f350-bbd1-43da-b16c-289f2705a04a service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:51.196332 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5498f350-bbd1-43da-b16c-289f2705a04a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:51.205597 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5498f350-bbd1-43da-b16c-289f2705a04a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:51.207025 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5498f350-bbd1-43da-b16c-289f2705a04a service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:51.217354 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1185/2353] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:58:51 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:58:51.456051 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2fd30-ac8b-4079-8108-cd0fc0c4a88d None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:51.462876 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2fd30-ac8b-4079-8108-cd0fc0c4a88d None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:51.467422 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731003c-7072-4eba-96f4-197170958d92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:51.468196 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731003c-7072-4eba-96f4-197170958d92 service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:51.468553 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731003c-7072-4eba-96f4-197170958d92 service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:51.468901 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731003c-7072-4eba-96f4-197170958d92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:51.532900 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731003c-7072-4eba-96f4-197170958d92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:51.535295 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731003c-7072-4eba-96f4-197170958d92 service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:51.543571 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1169/2354] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:58:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:58:53.144928 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5498f350-bbd1-43da-b16c-289f2705a04a service cinder] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:53.147167 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-5498f350-bbd1-43da-b16c-289f2705a04a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:53.224602 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5498f350-bbd1-43da-b16c-289f2705a04a service cinder] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:53.229919 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83a866-8b0e-4dec-aa6e-b1c036b3e191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:53.230553 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83a866-8b0e-4dec-aa6e-b1c036b3e191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:53.230861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83a866-8b0e-4dec-aa6e-b1c036b3e191 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:53.231133 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83a866-8b0e-4dec-aa6e-b1c036b3e191 admin admin] PATH_INFO: `/v3/users/d20fa4b6873f478fa8cd133c669f8cd7` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:53.232276 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba83a866-8b0e-4dec-aa6e-b1c036b3e191 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d20fa4b6873f478fa8cd133c669f8cd7)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:53.234190 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba83a866-8b0e-4dec-aa6e-b1c036b3e191 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:53.422096 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ba83a866-8b0e-4dec-aa6e-b1c036b3e191 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd20fa4b6873f478fa8cd133c669f8cd7'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:53.422826 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ba83a866-8b0e-4dec-aa6e-b1c036b3e191 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd20fa4b6873f478fa8cd133c669f8cd7'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:53.485650 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ba83a866-8b0e-4dec-aa6e-b1c036b3e191 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd20fa4b6873f478fa8cd133c669f8cd7'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:53.500634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ba83a866-8b0e-4dec-aa6e-b1c036b3e191 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd20fa4b6873f478fa8cd133c669f8cd7'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:53.501527 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ba83a866-8b0e-4dec-aa6e-b1c036b3e191 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd20fa4b6873f478fa8cd133c669f8cd7'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:53.517153 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1186/2355] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:58:53 2022] DELETE /identity/v3/users/d20fa4b6873f478fa8cd133c669f8cd7 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:53.844564 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731003c-7072-4eba-96f4-197170958d92 service cinder] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:53.856187 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731003c-7072-4eba-96f4-197170958d92 service cinder] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:53.918621 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecacf3ff-4970-4d8c-8e21-857b735230b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:53.919250 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecacf3ff-4970-4d8c-8e21-857b735230b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:53.919593 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecacf3ff-4970-4d8c-8e21-857b735230b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:53.919901 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecacf3ff-4970-4d8c-8e21-857b735230b5 admin admin] PATH_INFO: `/v3/projects/b4af2d53a1544e8a99cf19d4f103fbf2` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:53.921010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecacf3ff-4970-4d8c-8e21-857b735230b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4af2d53a1544e8a99cf19d4f103fbf2)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:53.922941 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecacf3ff-4970-4d8c-8e21-857b735230b5 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:54.090141 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-ecacf3ff-4970-4d8c-8e21-857b735230b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4af2d53a1544e8a99cf19d4f103fbf2', 'request_id': 'req-ecacf3ff-4970-4d8c-8e21-857b735230b5'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:54.156581 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-ecacf3ff-4970-4d8c-8e21-857b735230b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4af2d53a1544e8a99cf19d4f103fbf2', 'request_id': 'req-ecacf3ff-4970-4d8c-8e21-857b735230b5'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:54.162863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-ecacf3ff-4970-4d8c-8e21-857b735230b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4af2d53a1544e8a99cf19d4f103fbf2', 'request_id': 'req-ecacf3ff-4970-4d8c-8e21-857b735230b5'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:54.174981 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-ecacf3ff-4970-4d8c-8e21-857b735230b5 admin admin] The token cache is being invalidate because project b4af2d53a1544e8a99cf19d4f103fbf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:58:54.175318 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-ecacf3ff-4970-4d8c-8e21-857b735230b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:54.176106 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-ecacf3ff-4970-4d8c-8e21-857b735230b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:54.177914 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1170/2356] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:58:53 2022] DELETE /identity/v3/projects/b4af2d53a1544e8a99cf19d4f103fbf2 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:54.182602 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83a866-8b0e-4dec-aa6e-b1c036b3e191 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:54.184143 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-ba83a866-8b0e-4dec-aa6e-b1c036b3e191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:54.198425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba83a866-8b0e-4dec-aa6e-b1c036b3e191 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:54.234704 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:54.257879 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c9ed7-7ebd-4b75-a863-01c289fba02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:54.258621 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c9ed7-7ebd-4b75-a863-01c289fba02c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:54.258954 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c9ed7-7ebd-4b75-a863-01c289fba02c admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:54.259245 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c9ed7-7ebd-4b75-a863-01c289fba02c admin admin] PATH_INFO: `/v3/users/a76c572a801f4f6dac439ca4da0afbcd` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:54.260407 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65c9ed7-7ebd-4b75-a863-01c289fba02c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a76c572a801f4f6dac439ca4da0afbcd)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:54.262343 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65c9ed7-7ebd-4b75-a863-01c289fba02c admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:54.446138 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-b65c9ed7-7ebd-4b75-a863-01c289fba02c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a76c572a801f4f6dac439ca4da0afbcd'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:54.446749 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-b65c9ed7-7ebd-4b75-a863-01c289fba02c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a76c572a801f4f6dac439ca4da0afbcd'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:54.517531 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-b65c9ed7-7ebd-4b75-a863-01c289fba02c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a76c572a801f4f6dac439ca4da0afbcd'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:54.535042 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-b65c9ed7-7ebd-4b75-a863-01c289fba02c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a76c572a801f4f6dac439ca4da0afbcd'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:54.538117 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-b65c9ed7-7ebd-4b75-a863-01c289fba02c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a76c572a801f4f6dac439ca4da0afbcd'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:54.548860 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1187/2357] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:58:54 2022] DELETE /identity/v3/users/a76c572a801f4f6dac439ca4da0afbcd => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:54.798344 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecacf3ff-4970-4d8c-8e21-857b735230b5 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:54.799792 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-ecacf3ff-4970-4d8c-8e21-857b735230b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:54.812477 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecacf3ff-4970-4d8c-8e21-857b735230b5 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:54.820941 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:54.875163 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950b12d-4b52-4877-92c8-40c9a33f79f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:54.875915 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950b12d-4b52-4877-92c8-40c9a33f79f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:54.876149 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950b12d-4b52-4877-92c8-40c9a33f79f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:54.876422 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950b12d-4b52-4877-92c8-40c9a33f79f4 admin admin] PATH_INFO: `/v3/projects/283acce91bf34bfdb5e0c79df63846fe` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:54.877549 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950b12d-4b52-4877-92c8-40c9a33f79f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=283acce91bf34bfdb5e0c79df63846fe)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:54.879496 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950b12d-4b52-4877-92c8-40c9a33f79f4 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:54.994511 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f950b12d-4b52-4877-92c8-40c9a33f79f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '283acce91bf34bfdb5e0c79df63846fe', 'request_id': 'req-f950b12d-4b52-4877-92c8-40c9a33f79f4'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:55.051233 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f950b12d-4b52-4877-92c8-40c9a33f79f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '283acce91bf34bfdb5e0c79df63846fe', 'request_id': 'req-f950b12d-4b52-4877-92c8-40c9a33f79f4'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:55.057227 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f950b12d-4b52-4877-92c8-40c9a33f79f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '283acce91bf34bfdb5e0c79df63846fe', 'request_id': 'req-f950b12d-4b52-4877-92c8-40c9a33f79f4'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:55.068842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f950b12d-4b52-4877-92c8-40c9a33f79f4 admin admin] The token cache is being invalidate because project 283acce91bf34bfdb5e0c79df63846fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:58:55.069131 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f950b12d-4b52-4877-92c8-40c9a33f79f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:55.069782 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f950b12d-4b52-4877-92c8-40c9a33f79f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:55.071254 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1171/2358] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:58:54 2022] DELETE /identity/v3/projects/283acce91bf34bfdb5e0c79df63846fe => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:55.077482 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837861e-7e7d-47f1-8620-0303c4cf1476 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:55.077770 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837861e-7e7d-47f1-8620-0303c4cf1476 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:55.078084 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e837861e-7e7d-47f1-8620-0303c4cf1476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:55.098688 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-e837861e-7e7d-47f1-8620-0303c4cf1476 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:55.099926 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-e837861e-7e7d-47f1-8620-0303c4cf1476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:55.145141 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950b12d-4b52-4877-92c8-40c9a33f79f4 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:55.149410 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-f950b12d-4b52-4877-92c8-40c9a33f79f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:55.171074 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1188/2359] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:58:55.178965 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3c08d-40d3-45a7-b3d7-a2a9de37afb8 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:55.179246 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3c08d-40d3-45a7-b3d7-a2a9de37afb8 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:55.179517 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3c08d-40d3-45a7-b3d7-a2a9de37afb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:55.199784 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-31a3c08d-40d3-45a7-b3d7-a2a9de37afb8 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:55.201086 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-31a3c08d-40d3-45a7-b3d7-a2a9de37afb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:55.214078 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1189/2360] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:58:55.215102 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3c08d-40d3-45a7-b3d7-a2a9de37afb8 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:55.224378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3c08d-40d3-45a7-b3d7-a2a9de37afb8 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:55.229000 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c265b-e574-42ee-82a6-2edf919da877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:55.229673 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c265b-e574-42ee-82a6-2edf919da877 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:55.229922 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c265b-e574-42ee-82a6-2edf919da877 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:55.230192 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c265b-e574-42ee-82a6-2edf919da877 admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:55.230832 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c265b-e574-42ee-82a6-2edf919da877 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:58:55.231133 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c265b-e574-42ee-82a6-2edf919da877 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:55.231659 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950b12d-4b52-4877-92c8-40c9a33f79f4 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:55.232738 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c265b-e574-42ee-82a6-2edf919da877 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:55.236878 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6083b7f-3432-4ab8-8ba0-9c39517c985e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:55.237557 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6083b7f-3432-4ab8-8ba0-9c39517c985e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:55.237960 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6083b7f-3432-4ab8-8ba0-9c39517c985e admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:55.238238 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6083b7f-3432-4ab8-8ba0-9c39517c985e admin admin] PATH_INFO: `/v3/users/23b2b6e2ac4a48f697226b53f7a27a78` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:55.239373 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6083b7f-3432-4ab8-8ba0-9c39517c985e admin admin] RBAC: Authorizing `identity:delete_user(user_id=23b2b6e2ac4a48f697226b53f7a27a78)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:55.241302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6083b7f-3432-4ab8-8ba0-9c39517c985e admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:55.244624 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1190/2361] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:58:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 17 03:58:55.248913 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c265b-e574-42ee-82a6-2edf919da877 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:55.260096 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c265b-e574-42ee-82a6-2edf919da877 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:55.265464 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56447430-f9e2-457a-af7d-15344f15e26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:55.271265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56447430-f9e2-457a-af7d-15344f15e26d admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:55.271542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56447430-f9e2-457a-af7d-15344f15e26d admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:55.271843 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56447430-f9e2-457a-af7d-15344f15e26d admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:55.272501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56447430-f9e2-457a-af7d-15344f15e26d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:55.274425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56447430-f9e2-457a-af7d-15344f15e26d admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:55.355764 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1191/2362] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:58:55 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:58:55.359689 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56447430-f9e2-457a-af7d-15344f15e26d admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:55.378697 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:55.414167 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:55.416984 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-b6083b7f-3432-4ab8-8ba0-9c39517c985e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23b2b6e2ac4a48f697226b53f7a27a78'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:55.423703 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56447430-f9e2-457a-af7d-15344f15e26d admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:55.428647 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589b7f1-9d59-484e-a3a0-fbef01d1efaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:55.429306 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589b7f1-9d59-484e-a3a0-fbef01d1efaf admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:55.429586 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589b7f1-9d59-484e-a3a0-fbef01d1efaf admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:55.429859 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589b7f1-9d59-484e-a3a0-fbef01d1efaf admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:55.430495 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e589b7f1-9d59-484e-a3a0-fbef01d1efaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:55.432985 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e589b7f1-9d59-484e-a3a0-fbef01d1efaf admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:55.487872 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-b6083b7f-3432-4ab8-8ba0-9c39517c985e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23b2b6e2ac4a48f697226b53f7a27a78'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:55.501112 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-b6083b7f-3432-4ab8-8ba0-9c39517c985e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23b2b6e2ac4a48f697226b53f7a27a78'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:55.501773 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-b6083b7f-3432-4ab8-8ba0-9c39517c985e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23b2b6e2ac4a48f697226b53f7a27a78'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:55.502392 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-b6083b7f-3432-4ab8-8ba0-9c39517c985e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23b2b6e2ac4a48f697226b53f7a27a78'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:55.515933 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1172/2363] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:58:55 2022] DELETE /identity/v3/users/23b2b6e2ac4a48f697226b53f7a27a78 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:55.521469 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2403e-49cf-4643-9ba3-1bedae3d3811 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:55.525539 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2403e-49cf-4643-9ba3-1bedae3d3811 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:55.525539 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2403e-49cf-4643-9ba3-1bedae3d3811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:55.525824 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1192/2364] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:58:55 2022] POST /identity/v3/users => generated 283 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:58:55.528366 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589b7f1-9d59-484e-a3a0-fbef01d1efaf admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:55.530091 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-e589b7f1-9d59-484e-a3a0-fbef01d1efaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:55.542043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589b7f1-9d59-484e-a3a0-fbef01d1efaf admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:55.543215 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-0fd2403e-49cf-4643-9ba3-1bedae3d3811 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:55.545394 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-0fd2403e-49cf-4643-9ba3-1bedae3d3811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:55.603138 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d2887-851f-4b91-bda3-87d1c3899a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:55.603829 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d2887-851f-4b91-bda3-87d1c3899a82 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:55.604187 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d2887-851f-4b91-bda3-87d1c3899a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:55.604467 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0d2887-851f-4b91-bda3-87d1c3899a82 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:55.605116 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0d2887-851f-4b91-bda3-87d1c3899a82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:55.606863 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0d2887-851f-4b91-bda3-87d1c3899a82 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:55.609526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1173/2365] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:58:55.620310 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1193/2366] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:58:55.624525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2403e-49cf-4643-9ba3-1bedae3d3811 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:55.625402 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d2887-851f-4b91-bda3-87d1c3899a82 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:55.627121 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-de0d2887-851f-4b91-bda3-87d1c3899a82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:55.636670 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2403e-49cf-4643-9ba3-1bedae3d3811 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:55.641847 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696725e-2323-4558-994c-5cff9bec9b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:55.642477 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696725e-2323-4558-994c-5cff9bec9b37 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:55.642756 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696725e-2323-4558-994c-5cff9bec9b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:55.645920 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696725e-2323-4558-994c-5cff9bec9b37 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:55.645920 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5696725e-2323-4558-994c-5cff9bec9b37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:55.645920 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5696725e-2323-4558-994c-5cff9bec9b37 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:55.661873 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1174/2367] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:55 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:58:55.666022 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696725e-2323-4558-994c-5cff9bec9b37 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:55.676948 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696725e-2323-4558-994c-5cff9bec9b37 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:55.683195 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771dfc8b-6424-4d2d-aac0-fb43cedfed62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:55.683901 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771dfc8b-6424-4d2d-aac0-fb43cedfed62 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:55.684187 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771dfc8b-6424-4d2d-aac0-fb43cedfed62 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:55.684463 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771dfc8b-6424-4d2d-aac0-fb43cedfed62 admin admin] PATH_INFO: `/v3/projects/2a44d7e011fe4a53a863bfe9b513ab27/users/f0717a2295cb463e8f0645830551402d/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:55.691843 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0d2887-851f-4b91-bda3-87d1c3899a82 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:55.696002 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8defe8f3-7290-43cb-8ab9-2bf7099e7d79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:55.696617 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defe8f3-7290-43cb-8ab9-2bf7099e7d79 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:55.696897 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defe8f3-7290-43cb-8ab9-2bf7099e7d79 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:55.697806 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8defe8f3-7290-43cb-8ab9-2bf7099e7d79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:55.702075 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771dfc8b-6424-4d2d-aac0-fb43cedfed62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a44d7e011fe4a53a863bfe9b513ab27, user_id=f0717a2295cb463e8f0645830551402d, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:55.704828 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771dfc8b-6424-4d2d-aac0-fb43cedfed62 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:55.709246 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8defe8f3-7290-43cb-8ab9-2bf7099e7d79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:55.710717 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8defe8f3-7290-43cb-8ab9-2bf7099e7d79 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:55.721128 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1194/2368] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:58:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:58:55.800707 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1175/2369] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:58:55 2022] PUT /identity/v3/projects/2a44d7e011fe4a53a863bfe9b513ab27/users/f0717a2295cb463e8f0645830551402d/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:58:55.811679 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb135439-f28c-4292-8467-e60b252e09f4 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:55.812000 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb135439-f28c-4292-8467-e60b252e09f4 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:55.812277 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb135439-f28c-4292-8467-e60b252e09f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:55.841544 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-eb135439-f28c-4292-8467-e60b252e09f4 None None] MFA Rules not processed for user `f0717a2295cb463e8f0645830551402d`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:55.842766 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-eb135439-f28c-4292-8467-e60b252e09f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:55.877212 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:55.911907 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1195/2370] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:58:55.932482 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771dfc8b-6424-4d2d-aac0-fb43cedfed62 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:55.932482 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-771dfc8b-6424-4d2d-aac0-fb43cedfed62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:55.998113 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771dfc8b-6424-4d2d-aac0-fb43cedfed62 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:56.001779 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887e937-524f-45b9-a8e0-76e5bbac1651 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:56.002380 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887e937-524f-45b9-a8e0-76e5bbac1651 service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:56.002658 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887e937-524f-45b9-a8e0-76e5bbac1651 service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:56.002934 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8887e937-524f-45b9-a8e0-76e5bbac1651 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:56.011338 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8887e937-524f-45b9-a8e0-76e5bbac1651 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:56.012783 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8887e937-524f-45b9-a8e0-76e5bbac1651 service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:56.025244 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1176/2371] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:58:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:58:56.121194 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb135439-f28c-4292-8467-e60b252e09f4 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:56.122636 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-eb135439-f28c-4292-8467-e60b252e09f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:56.193408 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb135439-f28c-4292-8467-e60b252e09f4 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:56.198001 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc016bda-ae95-45ee-952e-18e950d01673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:56.198632 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc016bda-ae95-45ee-952e-18e950d01673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:56.198908 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc016bda-ae95-45ee-952e-18e950d01673 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:56.199178 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc016bda-ae95-45ee-952e-18e950d01673 admin admin] PATH_INFO: `/v3/projects/e0c6329adddf4cbcbfe6f87a5b94124a` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:56.200312 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc016bda-ae95-45ee-952e-18e950d01673 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0c6329adddf4cbcbfe6f87a5b94124a)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:56.202219 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc016bda-ae95-45ee-952e-18e950d01673 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:56.306207 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-cc016bda-ae95-45ee-952e-18e950d01673 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0c6329adddf4cbcbfe6f87a5b94124a', 'request_id': 'req-cc016bda-ae95-45ee-952e-18e950d01673'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:56.355952 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-cc016bda-ae95-45ee-952e-18e950d01673 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0c6329adddf4cbcbfe6f87a5b94124a', 'request_id': 'req-cc016bda-ae95-45ee-952e-18e950d01673'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:56.362023 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-cc016bda-ae95-45ee-952e-18e950d01673 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0c6329adddf4cbcbfe6f87a5b94124a', 'request_id': 'req-cc016bda-ae95-45ee-952e-18e950d01673'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:56.375050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-cc016bda-ae95-45ee-952e-18e950d01673 admin admin] The token cache is being invalidate because project e0c6329adddf4cbcbfe6f87a5b94124a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:58:56.375305 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-cc016bda-ae95-45ee-952e-18e950d01673 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:56.375915 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-cc016bda-ae95-45ee-952e-18e950d01673 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:56.377395 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1196/2372] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:58:56 2022] DELETE /identity/v3/projects/e0c6329adddf4cbcbfe6f87a5b94124a => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:56.381427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887e937-524f-45b9-a8e0-76e5bbac1651 service cinder] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:56.391958 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8887e937-524f-45b9-a8e0-76e5bbac1651 service cinder] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:56.420933 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:56.451038 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20687de-45a2-423d-9797-d556558b4601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:56.451710 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20687de-45a2-423d-9797-d556558b4601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:56.452011 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20687de-45a2-423d-9797-d556558b4601 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:56.452281 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20687de-45a2-423d-9797-d556558b4601 admin admin] PATH_INFO: `/v3/users/6f519e36ff6148cfaf102a12917bc689` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:56.453373 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20687de-45a2-423d-9797-d556558b4601 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f519e36ff6148cfaf102a12917bc689)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:56.455258 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20687de-45a2-423d-9797-d556558b4601 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:56.662147 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-d20687de-45a2-423d-9797-d556558b4601 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f519e36ff6148cfaf102a12917bc689'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:56.727881 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-d20687de-45a2-423d-9797-d556558b4601 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f519e36ff6148cfaf102a12917bc689'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:56.750246 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-d20687de-45a2-423d-9797-d556558b4601 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f519e36ff6148cfaf102a12917bc689'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:56.751023 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-d20687de-45a2-423d-9797-d556558b4601 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f519e36ff6148cfaf102a12917bc689'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:56.751946 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-d20687de-45a2-423d-9797-d556558b4601 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f519e36ff6148cfaf102a12917bc689'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:56.769755 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1177/2373] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:58:56 2022] DELETE /identity/v3/users/6f519e36ff6148cfaf102a12917bc689 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:56.792113 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc016bda-ae95-45ee-952e-18e950d01673 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:56.793842 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-cc016bda-ae95-45ee-952e-18e950d01673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:56.872043 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:56.899408 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc016bda-ae95-45ee-952e-18e950d01673 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:56.962335 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe3cfd3-bcf0-4e98-b9b5-b905684e044a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b500a113a2aa4a95875d7e094b917db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:56.963254 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3cfd3-bcf0-4e98-b9b5-b905684e044a service placement] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:56.963514 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3cfd3-bcf0-4e98-b9b5-b905684e044a service placement] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:56.963818 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe3cfd3-bcf0-4e98-b9b5-b905684e044a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:56.965572 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-7fe3cfd3-bcf0-4e98-b9b5-b905684e044a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:56.978148 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe3cfd3-bcf0-4e98-b9b5-b905684e044a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:56.979799 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe3cfd3-bcf0-4e98-b9b5-b905684e044a service placement] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:56.991863 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1197/2374] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:58:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:58:57.193357 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20687de-45a2-423d-9797-d556558b4601 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:57.193357 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-d20687de-45a2-423d-9797-d556558b4601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:57.264268 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20687de-45a2-423d-9797-d556558b4601 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:57.268925 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4bc604-00ac-4df9-ad66-4c6fb366c355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:57.273224 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4bc604-00ac-4df9-ad66-4c6fb366c355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:57.273224 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4bc604-00ac-4df9-ad66-4c6fb366c355 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:57.273224 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4bc604-00ac-4df9-ad66-4c6fb366c355 admin admin] PATH_INFO: `/v3/projects/faba5883c496433db39d4f723401082a` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:57.273224 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4bc604-00ac-4df9-ad66-4c6fb366c355 admin admin] RBAC: Authorizing `identity:delete_project(project_id=faba5883c496433db39d4f723401082a)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:57.273224 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4bc604-00ac-4df9-ad66-4c6fb366c355 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:57.378237 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-ae4bc604-00ac-4df9-ad66-4c6fb366c355 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faba5883c496433db39d4f723401082a', 'request_id': 'req-ae4bc604-00ac-4df9-ad66-4c6fb366c355'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:57.380406 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:57.474484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-ae4bc604-00ac-4df9-ad66-4c6fb366c355 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faba5883c496433db39d4f723401082a', 'request_id': 'req-ae4bc604-00ac-4df9-ad66-4c6fb366c355'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:57.480954 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-ae4bc604-00ac-4df9-ad66-4c6fb366c355 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faba5883c496433db39d4f723401082a', 'request_id': 'req-ae4bc604-00ac-4df9-ad66-4c6fb366c355'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:57.492172 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-ae4bc604-00ac-4df9-ad66-4c6fb366c355 admin admin] The token cache is being invalidate because project faba5883c496433db39d4f723401082a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:58:57.492458 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-ae4bc604-00ac-4df9-ad66-4c6fb366c355 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:57.493091 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-ae4bc604-00ac-4df9-ad66-4c6fb366c355 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:58:57.494613 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1178/2375] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:58:57 2022] DELETE /identity/v3/projects/faba5883c496433db39d4f723401082a => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:57.499425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53c2e3-572d-497b-97a3-cce625e31ae4 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:57.499973 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53c2e3-572d-497b-97a3-cce625e31ae4 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:57.499973 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53c2e3-572d-497b-97a3-cce625e31ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:57.521415 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-ba53c2e3-572d-497b-97a3-cce625e31ae4 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:57.522976 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-ba53c2e3-572d-497b-97a3-cce625e31ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:57.591372 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1198/2376] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:58:57.599275 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c3cf2-e4cd-46b2-bf31-1290e5500af8 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:57.599564 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c3cf2-e4cd-46b2-bf31-1290e5500af8 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:57.599917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c3cf2-e4cd-46b2-bf31-1290e5500af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:57.623496 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-cc8c3cf2-e4cd-46b2-bf31-1290e5500af8 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:57.624809 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-cc8c3cf2-e4cd-46b2-bf31-1290e5500af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:57.632615 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1179/2377] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:58:57.637432 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53c2e3-572d-497b-97a3-cce625e31ae4 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:57.647027 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53c2e3-572d-497b-97a3-cce625e31ae4 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:57.651556 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58974c5c-6ba4-4877-a206-780bd4884a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:57.652186 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58974c5c-6ba4-4877-a206-780bd4884a37 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:57.652465 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58974c5c-6ba4-4877-a206-780bd4884a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:57.652732 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58974c5c-6ba4-4877-a206-780bd4884a37 admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:57.653368 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58974c5c-6ba4-4877-a206-780bd4884a37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:58:57.653669 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58974c5c-6ba4-4877-a206-780bd4884a37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:57.655296 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58974c5c-6ba4-4877-a206-780bd4884a37 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:57.672422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1199/2378] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:58:57.675684 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c3cf2-e4cd-46b2-bf31-1290e5500af8 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:57.686691 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c3cf2-e4cd-46b2-bf31-1290e5500af8 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:57.691666 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548ad1a-9c9c-4107-a3fe-8565e0b87a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:57.692464 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548ad1a-9c9c-4107-a3fe-8565e0b87a4e admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:57.692794 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548ad1a-9c9c-4107-a3fe-8565e0b87a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:57.693137 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5548ad1a-9c9c-4107-a3fe-8565e0b87a4e admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:57.693856 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548ad1a-9c9c-4107-a3fe-8565e0b87a4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:57.696387 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5548ad1a-9c9c-4107-a3fe-8565e0b87a4e admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:57.799162 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1180/2379] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:58:57 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:58:57.800391 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58974c5c-6ba4-4877-a206-780bd4884a37 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:57.812220 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58974c5c-6ba4-4877-a206-780bd4884a37 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:57.875542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073465e4-ed6a-4ba7-a665-d2e44d682ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:57.876184 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073465e4-ed6a-4ba7-a665-d2e44d682ab8 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:57.876461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073465e4-ed6a-4ba7-a665-d2e44d682ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:57.876777 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073465e4-ed6a-4ba7-a665-d2e44d682ab8 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:57.877405 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073465e4-ed6a-4ba7-a665-d2e44d682ab8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:57.879337 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073465e4-ed6a-4ba7-a665-d2e44d682ab8 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:57.892397 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:58.024819 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1200/2380] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:58:57 2022] POST /identity/v3/users => generated 289 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:58:58.028829 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548ad1a-9c9c-4107-a3fe-8565e0b87a4e admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:58.038783 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5548ad1a-9c9c-4107-a3fe-8565e0b87a4e admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:58.044955 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545bca1-8605-43ef-8d11-26ef7a73c014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:58.045778 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545bca1-8605-43ef-8d11-26ef7a73c014 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:58.046143 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545bca1-8605-43ef-8d11-26ef7a73c014 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:58.046347 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545bca1-8605-43ef-8d11-26ef7a73c014 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:58.048455 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545bca1-8605-43ef-8d11-26ef7a73c014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:58.048660 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545bca1-8605-43ef-8d11-26ef7a73c014 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:58.061611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1181/2381] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:58:58.064028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073465e4-ed6a-4ba7-a665-d2e44d682ab8 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:58.074992 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073465e4-ed6a-4ba7-a665-d2e44d682ab8 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:58.079716 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415ed12-77d9-4efe-9003-9a52352fb0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:58.080342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415ed12-77d9-4efe-9003-9a52352fb0ad admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:58.080619 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415ed12-77d9-4efe-9003-9a52352fb0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:58.080890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415ed12-77d9-4efe-9003-9a52352fb0ad admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:58.081669 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415ed12-77d9-4efe-9003-9a52352fb0ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:58.083076 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415ed12-77d9-4efe-9003-9a52352fb0ad admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:58.094208 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1201/2382] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:58:58.098316 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545bca1-8605-43ef-8d11-26ef7a73c014 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:58.115500 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545bca1-8605-43ef-8d11-26ef7a73c014 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:58.120992 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5316ff4-a148-469a-a100-b46e6cf83c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:58.121786 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5316ff4-a148-469a-a100-b46e6cf83c25 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:58.122799 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5316ff4-a148-469a-a100-b46e6cf83c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:58.122799 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5316ff4-a148-469a-a100-b46e6cf83c25 admin admin] PATH_INFO: `/v3/projects/ede6946540ee486cbee59ecc18817fcd/users/6ee922928e1f467a86f7ea3f1736d20d/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:58.139415 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5316ff4-a148-469a-a100-b46e6cf83c25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ede6946540ee486cbee59ecc18817fcd, user_id=6ee922928e1f467a86f7ea3f1736d20d, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:58.142294 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5316ff4-a148-469a-a100-b46e6cf83c25 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:58.220032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1182/2383] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:58:58 2022] PUT /identity/v3/projects/ede6946540ee486cbee59ecc18817fcd/users/6ee922928e1f467a86f7ea3f1736d20d/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:58:58.231433 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201c671-87fa-4b5d-8315-d529c3bc4872 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:58.231724 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201c671-87fa-4b5d-8315-d529c3bc4872 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:58.232025 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201c671-87fa-4b5d-8315-d529c3bc4872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:58.264073 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-a201c671-87fa-4b5d-8315-d529c3bc4872 None None] MFA Rules not processed for user `6ee922928e1f467a86f7ea3f1736d20d`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:58.265342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-a201c671-87fa-4b5d-8315-d529c3bc4872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:58.330441 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1202/2384] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:58:58.335061 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5316ff4-a148-469a-a100-b46e6cf83c25 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:58.405117 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5316ff4-a148-469a-a100-b46e6cf83c25 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:58.409909 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0594399-4646-4e28-9167-a75d6ea4b7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:58.410627 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0594399-4646-4e28-9167-a75d6ea4b7cf admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:58.410911 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0594399-4646-4e28-9167-a75d6ea4b7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:58.411178 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0594399-4646-4e28-9167-a75d6ea4b7cf admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:58.411843 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0594399-4646-4e28-9167-a75d6ea4b7cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:58.413719 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0594399-4646-4e28-9167-a75d6ea4b7cf admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:58.493549 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1183/2385] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:58:58 2022] POST /identity/v3/projects => generated 375 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:58:58.497790 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201c671-87fa-4b5d-8315-d529c3bc4872 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:58.510646 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201c671-87fa-4b5d-8315-d529c3bc4872 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:58.570479 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46804bdc-5438-4907-b7c9-6d132de3f8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:58.571090 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46804bdc-5438-4907-b7c9-6d132de3f8ac admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:58.571374 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46804bdc-5438-4907-b7c9-6d132de3f8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:58.571723 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46804bdc-5438-4907-b7c9-6d132de3f8ac admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:58.572422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46804bdc-5438-4907-b7c9-6d132de3f8ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:58.574394 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46804bdc-5438-4907-b7c9-6d132de3f8ac admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:58.659694 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1203/2386] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:58:58 2022] POST /identity/v3/users => generated 294 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:58:58.663995 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0594399-4646-4e28-9167-a75d6ea4b7cf admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:58.674948 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0594399-4646-4e28-9167-a75d6ea4b7cf admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:58.679840 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982a0d7-e7c6-4a03-b90b-136596ffba09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:58.680473 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982a0d7-e7c6-4a03-b90b-136596ffba09 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:58.680761 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982a0d7-e7c6-4a03-b90b-136596ffba09 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:58.681041 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6982a0d7-e7c6-4a03-b90b-136596ffba09 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:58.681698 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982a0d7-e7c6-4a03-b90b-136596ffba09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:58.683505 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6982a0d7-e7c6-4a03-b90b-136596ffba09 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:58.699641 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1184/2387] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:58:58.700906 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46804bdc-5438-4907-b7c9-6d132de3f8ac admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:58.731766 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46804bdc-5438-4907-b7c9-6d132de3f8ac admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:58.737123 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033968fc-01a3-4558-acf3-e36453a74efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:58.737813 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033968fc-01a3-4558-acf3-e36453a74efe admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:58.738088 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033968fc-01a3-4558-acf3-e36453a74efe admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:58.738359 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033968fc-01a3-4558-acf3-e36453a74efe admin admin] PATH_INFO: `/v3/projects/ddcd21a4dda14f489088dd2be9982a07/users/2c470718caa44deca6f100a2758c3cfc/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:58.758302 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033968fc-01a3-4558-acf3-e36453a74efe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddcd21a4dda14f489088dd2be9982a07, user_id=2c470718caa44deca6f100a2758c3cfc, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:58.761507 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033968fc-01a3-4558-acf3-e36453a74efe admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:58.863866 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1204/2388] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:58:58 2022] PUT /identity/v3/projects/ddcd21a4dda14f489088dd2be9982a07/users/2c470718caa44deca6f100a2758c3cfc/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:58.867887 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982a0d7-e7c6-4a03-b90b-136596ffba09 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:58.878024 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6982a0d7-e7c6-4a03-b90b-136596ffba09 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:58.923682 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:58.935220 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8cd229-fb0d-4f09-9df8-814c2212edd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:58.935877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cd229-fb0d-4f09-9df8-814c2212edd1 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:58.936156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cd229-fb0d-4f09-9df8-814c2212edd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:58.936424 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8cd229-fb0d-4f09-9df8-814c2212edd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:58.937056 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8cd229-fb0d-4f09-9df8-814c2212edd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:58.938709 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8cd229-fb0d-4f09-9df8-814c2212edd1 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:58.948992 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1185/2389] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:58:58 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:58:58.953392 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033968fc-01a3-4558-acf3-e36453a74efe admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:58.964838 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033968fc-01a3-4558-acf3-e36453a74efe admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:58.969718 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028643d6-124e-4aa5-82df-16b2fcfeef9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:58.970369 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028643d6-124e-4aa5-82df-16b2fcfeef9b admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:58.970667 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028643d6-124e-4aa5-82df-16b2fcfeef9b admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:58.970934 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028643d6-124e-4aa5-82df-16b2fcfeef9b admin admin] PATH_INFO: `/v3/projects/ddcd21a4dda14f489088dd2be9982a07/users/2c470718caa44deca6f100a2758c3cfc/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:58.972869 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028643d6-124e-4aa5-82df-16b2fcfeef9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddcd21a4dda14f489088dd2be9982a07, user_id=2c470718caa44deca6f100a2758c3cfc, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:58.975490 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028643d6-124e-4aa5-82df-16b2fcfeef9b admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:58.985724 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1205/2390] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:58:58 2022] PUT /identity/v3/projects/ddcd21a4dda14f489088dd2be9982a07/users/2c470718caa44deca6f100a2758c3cfc/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:58:58.998040 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246774d-3b75-4b58-9c02-e6b41dd435a5 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:58.998040 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246774d-3b75-4b58-9c02-e6b41dd435a5 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:58.998040 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246774d-3b75-4b58-9c02-e6b41dd435a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:59.026609 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-5246774d-3b75-4b58-9c02-e6b41dd435a5 None None] MFA Rules not processed for user `2c470718caa44deca6f100a2758c3cfc`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:58:59.027880 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-5246774d-3b75-4b58-9c02-e6b41dd435a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:59.096609 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1186/2391] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:58:58 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:58:59.107313 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028643d6-124e-4aa5-82df-16b2fcfeef9b admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:58:59.109128 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-028643d6-124e-4aa5-82df-16b2fcfeef9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:58:59.138167 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:58:59.169174 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028643d6-124e-4aa5-82df-16b2fcfeef9b admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:58:59.172854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d02c9-dd42-4ec2-b0e9-6ed721e113c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:58:59.173456 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d02c9-dd42-4ec2-b0e9-6ed721e113c3 service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:58:59.173739 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d02c9-dd42-4ec2-b0e9-6ed721e113c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:58:59.174005 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d02c9-dd42-4ec2-b0e9-6ed721e113c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:58:59.231765 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419d02c9-dd42-4ec2-b0e9-6ed721e113c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:58:59.233209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419d02c9-dd42-4ec2-b0e9-6ed721e113c3 service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:58:59.242172 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1206/2392] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:58:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:59:05.612486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246774d-3b75-4b58-9c02-e6b41dd435a5 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:05.613976 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-5246774d-3b75-4b58-9c02-e6b41dd435a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:05.626054 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246774d-3b75-4b58-9c02-e6b41dd435a5 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:05.629845 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce0985f-8cdc-4cb5-903b-93e5bceb30c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:05.630436 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0985f-8cdc-4cb5-903b-93e5bceb30c7 service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:05.630734 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0985f-8cdc-4cb5-903b-93e5bceb30c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:05.631018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce0985f-8cdc-4cb5-903b-93e5bceb30c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:05.641539 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce0985f-8cdc-4cb5-903b-93e5bceb30c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:05.643128 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce0985f-8cdc-4cb5-903b-93e5bceb30c7 service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:05.653206 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1187/2393] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:59:05 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:59:09.348656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=37 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:15.648392 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=47 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:16.354685 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d02c9-dd42-4ec2-b0e9-6ed721e113c3 service cinder] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:16.356131 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-419d02c9-dd42-4ec2-b0e9-6ed721e113c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:16.422996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d02c9-dd42-4ec2-b0e9-6ed721e113c3 service cinder] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:16.427511 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790b89d-1528-4969-b4c7-11f7df585bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:16.428169 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790b89d-1528-4969-b4c7-11f7df585bea admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:16.428449 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790b89d-1528-4969-b4c7-11f7df585bea admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:16.428719 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790b89d-1528-4969-b4c7-11f7df585bea admin admin] PATH_INFO: `/v3/users/e50692a6da234d2abaee5cdfd0cfd268` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:16.429848 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790b89d-1528-4969-b4c7-11f7df585bea admin admin] RBAC: Authorizing `identity:delete_user(user_id=e50692a6da234d2abaee5cdfd0cfd268)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:16.431763 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790b89d-1528-4969-b4c7-11f7df585bea admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:16.599467 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2790b89d-1528-4969-b4c7-11f7df585bea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e50692a6da234d2abaee5cdfd0cfd268'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:16.600182 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2790b89d-1528-4969-b4c7-11f7df585bea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e50692a6da234d2abaee5cdfd0cfd268'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:16.698623 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2790b89d-1528-4969-b4c7-11f7df585bea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e50692a6da234d2abaee5cdfd0cfd268'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:16.712611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2790b89d-1528-4969-b4c7-11f7df585bea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e50692a6da234d2abaee5cdfd0cfd268'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:16.713536 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2790b89d-1528-4969-b4c7-11f7df585bea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e50692a6da234d2abaee5cdfd0cfd268'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:16.726259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1207/2394] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:59:16 2022] DELETE /identity/v3/users/e50692a6da234d2abaee5cdfd0cfd268 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:16.730746 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7227e-e9c1-4d1b-afb3-b0ed8185d04b None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:16.731246 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7227e-e9c1-4d1b-afb3-b0ed8185d04b None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:16.731367 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7227e-e9c1-4d1b-afb3-b0ed8185d04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:16.749756 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-8cb7227e-e9c1-4d1b-afb3-b0ed8185d04b None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:16.750947 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-8cb7227e-e9c1-4d1b-afb3-b0ed8185d04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:16.807811 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1188/2395] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:59:16.817325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790b89d-1528-4969-b4c7-11f7df585bea admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:16.818832 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-2790b89d-1528-4969-b4c7-11f7df585bea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:16.895810 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790b89d-1528-4969-b4c7-11f7df585bea admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:16.899371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b0441-4ef3-4d4a-ac28-df9159532d21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:16.900002 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b0441-4ef3-4d4a-ac28-df9159532d21 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:16.900287 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b0441-4ef3-4d4a-ac28-df9159532d21 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:16.900562 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b0441-4ef3-4d4a-ac28-df9159532d21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:16.909993 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b0441-4ef3-4d4a-ac28-df9159532d21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:16.911408 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b0441-4ef3-4d4a-ac28-df9159532d21 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:16.920933 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1208/2396] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:59:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:59:17.321071 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb7227e-e9c1-4d1b-afb3-b0ed8185d04b None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:17.322677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-8cb7227e-e9c1-4d1b-afb3-b0ed8185d04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:17.337636 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb7227e-e9c1-4d1b-afb3-b0ed8185d04b None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:17.343820 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d58aac-deca-4f8b-a432-981a7ae60e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:17.344286 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d58aac-deca-4f8b-a432-981a7ae60e88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:17.344568 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d58aac-deca-4f8b-a432-981a7ae60e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:17.344845 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d58aac-deca-4f8b-a432-981a7ae60e88 admin admin] PATH_INFO: `/v3/projects/f613f0eb6cd14f9daadf8694b60f6aa8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:17.345970 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d58aac-deca-4f8b-a432-981a7ae60e88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f613f0eb6cd14f9daadf8694b60f6aa8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:17.347985 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d58aac-deca-4f8b-a432-981a7ae60e88 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:17.451442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-01d58aac-deca-4f8b-a432-981a7ae60e88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f613f0eb6cd14f9daadf8694b60f6aa8', 'request_id': 'req-01d58aac-deca-4f8b-a432-981a7ae60e88'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:17.519645 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-01d58aac-deca-4f8b-a432-981a7ae60e88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f613f0eb6cd14f9daadf8694b60f6aa8', 'request_id': 'req-01d58aac-deca-4f8b-a432-981a7ae60e88'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:17.525888 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-01d58aac-deca-4f8b-a432-981a7ae60e88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f613f0eb6cd14f9daadf8694b60f6aa8', 'request_id': 'req-01d58aac-deca-4f8b-a432-981a7ae60e88'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:17.537005 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-01d58aac-deca-4f8b-a432-981a7ae60e88 admin admin] The token cache is being invalidate because project f613f0eb6cd14f9daadf8694b60f6aa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:59:17.537288 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-01d58aac-deca-4f8b-a432-981a7ae60e88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:17.537947 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-01d58aac-deca-4f8b-a432-981a7ae60e88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:17.539422 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1189/2397] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:59:17 2022] DELETE /identity/v3/projects/f613f0eb6cd14f9daadf8694b60f6aa8 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:17.544715 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea88fb2-1102-425a-84b5-e0c88dc4b730 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:17.545232 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea88fb2-1102-425a-84b5-e0c88dc4b730 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:17.545361 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea88fb2-1102-425a-84b5-e0c88dc4b730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:17.552903 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:17.566103 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-9ea88fb2-1102-425a-84b5-e0c88dc4b730 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:17.566798 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-9ea88fb2-1102-425a-84b5-e0c88dc4b730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:17.626085 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1209/2398] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:59:17.633879 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a4f89-fd3d-4c0d-ab3c-7dced8634a2a None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:17.634195 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a4f89-fd3d-4c0d-ab3c-7dced8634a2a None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:17.634472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9a4f89-fd3d-4c0d-ab3c-7dced8634a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:17.654191 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-1e9a4f89-fd3d-4c0d-ab3c-7dced8634a2a None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:17.655386 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-1e9a4f89-fd3d-4c0d-ab3c-7dced8634a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:17.663257 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1190/2399] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:59:17.667352 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea88fb2-1102-425a-84b5-e0c88dc4b730 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:17.677525 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea88fb2-1102-425a-84b5-e0c88dc4b730 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:17.682200 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59aa29-8228-4606-8c4f-0982c6b9846f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:17.682915 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59aa29-8228-4606-8c4f-0982c6b9846f admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:17.683645 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59aa29-8228-4606-8c4f-0982c6b9846f admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:17.683645 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59aa29-8228-4606-8c4f-0982c6b9846f admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:17.684154 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59aa29-8228-4606-8c4f-0982c6b9846f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:59:17.685096 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59aa29-8228-4606-8c4f-0982c6b9846f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:17.686701 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59aa29-8228-4606-8c4f-0982c6b9846f admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:17.699205 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1210/2400] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:59:17.703515 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a4f89-fd3d-4c0d-ab3c-7dced8634a2a None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:17.712967 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9a4f89-fd3d-4c0d-ab3c-7dced8634a2a None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:17.717994 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c3bc0-464b-467c-a7de-e03dd11e8fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:17.718626 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c3bc0-464b-467c-a7de-e03dd11e8fc6 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:17.718908 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c3bc0-464b-467c-a7de-e03dd11e8fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:17.719184 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c3bc0-464b-467c-a7de-e03dd11e8fc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:17.719865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c3bc0-464b-467c-a7de-e03dd11e8fc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:17.721790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3c3bc0-464b-467c-a7de-e03dd11e8fc6 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:17.849523 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1191/2401] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:59:17 2022] POST /identity/v3/projects => generated 385 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:59:17.853278 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59aa29-8228-4606-8c4f-0982c6b9846f admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:17.864044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59aa29-8228-4606-8c4f-0982c6b9846f admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:17.919702 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3622ccf-76db-472d-a1d2-8287e0d5d43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:17.920334 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3622ccf-76db-472d-a1d2-8287e0d5d43d admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:17.920615 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3622ccf-76db-472d-a1d2-8287e0d5d43d admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:17.920890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3622ccf-76db-472d-a1d2-8287e0d5d43d admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:17.921516 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3622ccf-76db-472d-a1d2-8287e0d5d43d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:17.923414 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3622ccf-76db-472d-a1d2-8287e0d5d43d admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:18.030869 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1211/2402] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:59:17 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:59:18.035110 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c3bc0-464b-467c-a7de-e03dd11e8fc6 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:18.046095 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c3bc0-464b-467c-a7de-e03dd11e8fc6 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:18.050709 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4ac4a-51b0-4130-a1ea-6d896709308f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:18.051305 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4ac4a-51b0-4130-a1ea-6d896709308f admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:18.051617 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4ac4a-51b0-4130-a1ea-6d896709308f admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:18.051934 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4ac4a-51b0-4130-a1ea-6d896709308f admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:18.052578 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4ac4a-51b0-4130-a1ea-6d896709308f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:18.054160 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4ac4a-51b0-4130-a1ea-6d896709308f admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:18.064007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1192/2403] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:59:18.067824 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3622ccf-76db-472d-a1d2-8287e0d5d43d admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:18.077648 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3622ccf-76db-472d-a1d2-8287e0d5d43d admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:18.082179 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da6ece-4962-4512-819c-cc1471eb1f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:18.082769 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da6ece-4962-4512-819c-cc1471eb1f58 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:18.083051 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da6ece-4962-4512-819c-cc1471eb1f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:18.083323 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da6ece-4962-4512-819c-cc1471eb1f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:18.084003 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da6ece-4962-4512-819c-cc1471eb1f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:18.085571 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64da6ece-4962-4512-819c-cc1471eb1f58 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:18.096731 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1212/2404] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:59:18.100605 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4ac4a-51b0-4130-a1ea-6d896709308f admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:18.111157 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4ac4a-51b0-4130-a1ea-6d896709308f admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:18.115686 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff4edd6-3e1c-4629-a9dd-8bf699de8dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:18.116354 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4edd6-3e1c-4629-a9dd-8bf699de8dcb admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:18.116637 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4edd6-3e1c-4629-a9dd-8bf699de8dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:18.116946 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff4edd6-3e1c-4629-a9dd-8bf699de8dcb admin admin] PATH_INFO: `/v3/projects/eb26eaeda2e24f5fb7a80c2008f48209/users/eae797065d574847b923a7ac71b8c804/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:18.131124 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff4edd6-3e1c-4629-a9dd-8bf699de8dcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb26eaeda2e24f5fb7a80c2008f48209, user_id=eae797065d574847b923a7ac71b8c804, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:18.133878 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff4edd6-3e1c-4629-a9dd-8bf699de8dcb admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:18.190194 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1193/2405] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:59:18 2022] PUT /identity/v3/projects/eb26eaeda2e24f5fb7a80c2008f48209/users/eae797065d574847b923a7ac71b8c804/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:18.202235 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656ce65-fba1-4ec2-99f3-85ad0b3e79f2 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:18.202522 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656ce65-fba1-4ec2-99f3-85ad0b3e79f2 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:18.202797 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656ce65-fba1-4ec2-99f3-85ad0b3e79f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:18.231451 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-3656ce65-fba1-4ec2-99f3-85ad0b3e79f2 None None] MFA Rules not processed for user `eae797065d574847b923a7ac71b8c804`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:18.232703 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-3656ce65-fba1-4ec2-99f3-85ad0b3e79f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:18.290391 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:18.295448 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1213/2406] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:59:18.303835 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff4edd6-3e1c-4629-a9dd-8bf699de8dcb admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:18.320381 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:18.362416 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff4edd6-3e1c-4629-a9dd-8bf699de8dcb admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:18.367063 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2ff23-f0dd-4ace-9fa1-d60de0a19bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:18.367668 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2ff23-f0dd-4ace-9fa1-d60de0a19bc1 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:18.367989 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2ff23-f0dd-4ace-9fa1-d60de0a19bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:18.368264 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d2ff23-f0dd-4ace-9fa1-d60de0a19bc1 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:18.368903 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d2ff23-f0dd-4ace-9fa1-d60de0a19bc1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:18.370780 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d2ff23-f0dd-4ace-9fa1-d60de0a19bc1 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:18.459857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1194/2407] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:59:18 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:59:18.464169 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3656ce65-fba1-4ec2-99f3-85ad0b3e79f2 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:18.475546 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3656ce65-fba1-4ec2-99f3-85ad0b3e79f2 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:18.533493 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49841389-ea10-41ac-ba38-62535a1a7140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:18.534117 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49841389-ea10-41ac-ba38-62535a1a7140 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:18.534398 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49841389-ea10-41ac-ba38-62535a1a7140 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:18.534672 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49841389-ea10-41ac-ba38-62535a1a7140 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:18.535327 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49841389-ea10-41ac-ba38-62535a1a7140 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:18.537377 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49841389-ea10-41ac-ba38-62535a1a7140 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:18.642155 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1214/2408] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:59:18 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:59:18.646121 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2ff23-f0dd-4ace-9fa1-d60de0a19bc1 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:18.656392 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d2ff23-f0dd-4ace-9fa1-d60de0a19bc1 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:18.661061 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8b2ba-9e09-4295-a7bc-53e8a814ae87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:18.661662 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8b2ba-9e09-4295-a7bc-53e8a814ae87 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:18.661944 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8b2ba-9e09-4295-a7bc-53e8a814ae87 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:18.662218 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b8b2ba-9e09-4295-a7bc-53e8a814ae87 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:18.662857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8b2ba-9e09-4295-a7bc-53e8a814ae87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:18.664484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b8b2ba-9e09-4295-a7bc-53e8a814ae87 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:18.674669 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1195/2409] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:59:18.678629 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49841389-ea10-41ac-ba38-62535a1a7140 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:18.689055 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49841389-ea10-41ac-ba38-62535a1a7140 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:18.693634 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ffb715-125c-40f2-b2aa-9895fdb9a7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:18.694226 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffb715-125c-40f2-b2aa-9895fdb9a7c4 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:18.694505 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffb715-125c-40f2-b2aa-9895fdb9a7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:18.694775 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffb715-125c-40f2-b2aa-9895fdb9a7c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:18.695411 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ffb715-125c-40f2-b2aa-9895fdb9a7c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:18.697027 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ffb715-125c-40f2-b2aa-9895fdb9a7c4 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:18.707547 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1215/2410] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:59:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:59:18.711604 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8b2ba-9e09-4295-a7bc-53e8a814ae87 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:18.721872 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b8b2ba-9e09-4295-a7bc-53e8a814ae87 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:18.726437 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4a7a4-f13e-4f79-9f58-aff934da2a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:18.727082 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4a7a4-f13e-4f79-9f58-aff934da2a4e admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:18.727360 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4a7a4-f13e-4f79-9f58-aff934da2a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:18.727633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4a7a4-f13e-4f79-9f58-aff934da2a4e admin admin] PATH_INFO: `/v3/projects/3ec8abcd494a47008706ac384217b971/users/b0d51c2125cf4f0ca1c60a8de3b727d9/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:18.742164 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4a7a4-f13e-4f79-9f58-aff934da2a4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ec8abcd494a47008706ac384217b971, user_id=b0d51c2125cf4f0ca1c60a8de3b727d9, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:18.744813 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4a7a4-f13e-4f79-9f58-aff934da2a4e admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:18.824499 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1196/2411] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:59:18 2022] PUT /identity/v3/projects/3ec8abcd494a47008706ac384217b971/users/b0d51c2125cf4f0ca1c60a8de3b727d9/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:18.835053 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53ed5e-3b47-4b4c-b4b6-a291d131df7c None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:18.835331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53ed5e-3b47-4b4c-b4b6-a291d131df7c None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:18.835600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53ed5e-3b47-4b4c-b4b6-a291d131df7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:18.862871 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-8f53ed5e-3b47-4b4c-b4b6-a291d131df7c None None] MFA Rules not processed for user `b0d51c2125cf4f0ca1c60a8de3b727d9`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:18.864110 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-8f53ed5e-3b47-4b4c-b4b6-a291d131df7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:18.935342 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1216/2412] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:59:18.939525 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4a7a4-f13e-4f79-9f58-aff934da2a4e admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:19.004194 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4a7a4-f13e-4f79-9f58-aff934da2a4e admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:19.008904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a73022a-32cf-4b96-ae16-df009be6b1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:19.009506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73022a-32cf-4b96-ae16-df009be6b1aa admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:19.009805 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73022a-32cf-4b96-ae16-df009be6b1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:19.010078 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73022a-32cf-4b96-ae16-df009be6b1aa admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:19.010740 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a73022a-32cf-4b96-ae16-df009be6b1aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:19.012738 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a73022a-32cf-4b96-ae16-df009be6b1aa admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:19.145456 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1197/2413] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:59:18 2022] POST /identity/v3/projects => generated 383 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:59:19.149322 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53ed5e-3b47-4b4c-b4b6-a291d131df7c None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:19.160310 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53ed5e-3b47-4b4c-b4b6-a291d131df7c None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:19.217177 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ba0ab-bdac-4e00-8eb5-94e34354f993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:19.217787 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ba0ab-bdac-4e00-8eb5-94e34354f993 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:19.218081 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ba0ab-bdac-4e00-8eb5-94e34354f993 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:19.218359 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2ba0ab-bdac-4e00-8eb5-94e34354f993 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:19.219004 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ba0ab-bdac-4e00-8eb5-94e34354f993 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:19.220970 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2ba0ab-bdac-4e00-8eb5-94e34354f993 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:19.232464 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:19.345576 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1217/2414] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:59:19 2022] POST /identity/v3/users => generated 298 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:59:19.349570 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a73022a-32cf-4b96-ae16-df009be6b1aa admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:19.360658 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a73022a-32cf-4b96-ae16-df009be6b1aa admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:19.366760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e19bd-c8cf-4a82-ad24-c07e49384f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:19.366760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e19bd-c8cf-4a82-ad24-c07e49384f27 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:19.366760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e19bd-c8cf-4a82-ad24-c07e49384f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:19.366760 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e19bd-c8cf-4a82-ad24-c07e49384f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:19.367305 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e19bd-c8cf-4a82-ad24-c07e49384f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:19.368913 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e19bd-c8cf-4a82-ad24-c07e49384f27 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:19.379390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1198/2415] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:59:19.387193 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ba0ab-bdac-4e00-8eb5-94e34354f993 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:19.397549 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2ba0ab-bdac-4e00-8eb5-94e34354f993 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:19.402045 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d07db3-b12a-4d86-9174-d16a6553e014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:19.406800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d07db3-b12a-4d86-9174-d16a6553e014 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:19.406800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d07db3-b12a-4d86-9174-d16a6553e014 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:19.406800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d07db3-b12a-4d86-9174-d16a6553e014 admin admin] PATH_INFO: `/v3/projects/ae4b98ea4f204ead8649e08ea59fa03b/users/461e2d6e0e7141df875674822fe02210/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:19.418332 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d07db3-b12a-4d86-9174-d16a6553e014 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae4b98ea4f204ead8649e08ea59fa03b, user_id=461e2d6e0e7141df875674822fe02210, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:19.421775 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d07db3-b12a-4d86-9174-d16a6553e014 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:19.492896 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1218/2416] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:59:19 2022] PUT /identity/v3/projects/ae4b98ea4f204ead8649e08ea59fa03b/users/461e2d6e0e7141df875674822fe02210/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:19.496918 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e19bd-c8cf-4a82-ad24-c07e49384f27 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:19.506792 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e19bd-c8cf-4a82-ad24-c07e49384f27 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:19.548882 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:19.564052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4aab42-b1c9-41fc-a105-ea43b0755379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:19.564672 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4aab42-b1c9-41fc-a105-ea43b0755379 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:19.570060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4aab42-b1c9-41fc-a105-ea43b0755379 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:19.570060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4aab42-b1c9-41fc-a105-ea43b0755379 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:19.570060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4aab42-b1c9-41fc-a105-ea43b0755379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:19.570060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4aab42-b1c9-41fc-a105-ea43b0755379 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:19.582215 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1199/2417] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:59:19 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:59:19.582900 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d07db3-b12a-4d86-9174-d16a6553e014 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:19.599089 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d07db3-b12a-4d86-9174-d16a6553e014 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:19.600190 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882aad3a-edb1-44d3-b666-9862af8cd08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:19.600934 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882aad3a-edb1-44d3-b666-9862af8cd08f admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:19.601283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882aad3a-edb1-44d3-b666-9862af8cd08f admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:19.601623 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882aad3a-edb1-44d3-b666-9862af8cd08f admin admin] PATH_INFO: `/v3/projects/ae4b98ea4f204ead8649e08ea59fa03b/users/461e2d6e0e7141df875674822fe02210/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:19.603795 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882aad3a-edb1-44d3-b666-9862af8cd08f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae4b98ea4f204ead8649e08ea59fa03b, user_id=461e2d6e0e7141df875674822fe02210, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:19.607250 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882aad3a-edb1-44d3-b666-9862af8cd08f admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:19.619477 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1219/2418] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:59:19 2022] PUT /identity/v3/projects/ae4b98ea4f204ead8649e08ea59fa03b/users/461e2d6e0e7141df875674822fe02210/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:19.629907 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0bce5-f373-4952-a108-f0ab5b8e8f41 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:19.630195 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0bce5-f373-4952-a108-f0ab5b8e8f41 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:19.630469 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0bce5-f373-4952-a108-f0ab5b8e8f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:19.659338 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-31b0bce5-f373-4952-a108-f0ab5b8e8f41 None None] MFA Rules not processed for user `461e2d6e0e7141df875674822fe02210`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:19.660500 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-31b0bce5-f373-4952-a108-f0ab5b8e8f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:19.728762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1200/2419] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:19 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:59:19.748010 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882aad3a-edb1-44d3-b666-9862af8cd08f admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:19.753096 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-882aad3a-edb1-44d3-b666-9862af8cd08f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:19.828027 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882aad3a-edb1-44d3-b666-9862af8cd08f admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:19.831682 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6faae0-ce88-4972-bb97-2ba2e9a79c92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:19.832305 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6faae0-ce88-4972-bb97-2ba2e9a79c92 service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:19.832646 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6faae0-ce88-4972-bb97-2ba2e9a79c92 service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:19.832900 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6faae0-ce88-4972-bb97-2ba2e9a79c92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:19.895964 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6faae0-ce88-4972-bb97-2ba2e9a79c92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:19.897441 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6faae0-ce88-4972-bb97-2ba2e9a79c92 service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:19.907226 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1220/2420] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:59:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:59:21.920771 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0bce5-f373-4952-a108-f0ab5b8e8f41 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:21.922236 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-31b0bce5-f373-4952-a108-f0ab5b8e8f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:21.933831 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0bce5-f373-4952-a108-f0ab5b8e8f41 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:21.938112 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e49af1-1a98-4f0c-86a1-edd439d1f413 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:21.938710 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e49af1-1a98-4f0c-86a1-edd439d1f413 service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:21.938990 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e49af1-1a98-4f0c-86a1-edd439d1f413 service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:21.939263 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e49af1-1a98-4f0c-86a1-edd439d1f413 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:21.997486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e49af1-1a98-4f0c-86a1-edd439d1f413 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:21.998970 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e49af1-1a98-4f0c-86a1-edd439d1f413 service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:22.009124 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1201/2421] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:59:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:59:25.448714 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6faae0-ce88-4972-bb97-2ba2e9a79c92 service cinder] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:25.457573 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6faae0-ce88-4972-bb97-2ba2e9a79c92 service cinder] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:25.461274 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552159f5-25c9-4f5f-a1b9-06c38890119a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:25.461904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552159f5-25c9-4f5f-a1b9-06c38890119a service cinder] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:25.462183 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552159f5-25c9-4f5f-a1b9-06c38890119a service cinder] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:25.462450 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552159f5-25c9-4f5f-a1b9-06c38890119a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:25.473205 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552159f5-25c9-4f5f-a1b9-06c38890119a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:25.474687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552159f5-25c9-4f5f-a1b9-06c38890119a service cinder] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:25.484919 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1221/2422] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:59:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:59:32.004441 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=71 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:32.208174 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e49af1-1a98-4f0c-86a1-edd439d1f413 service cinder] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:32.271508 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e49af1-1a98-4f0c-86a1-edd439d1f413 service cinder] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:32.276274 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89118289-b5cb-405a-9f12-8c23f1c3238e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:32.276906 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89118289-b5cb-405a-9f12-8c23f1c3238e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:32.277186 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89118289-b5cb-405a-9f12-8c23f1c3238e admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:32.277456 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89118289-b5cb-405a-9f12-8c23f1c3238e admin admin] PATH_INFO: `/v3/users/eae797065d574847b923a7ac71b8c804` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:32.278582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89118289-b5cb-405a-9f12-8c23f1c3238e admin admin] RBAC: Authorizing `identity:delete_user(user_id=eae797065d574847b923a7ac71b8c804)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:32.280593 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89118289-b5cb-405a-9f12-8c23f1c3238e admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:32.526499 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-89118289-b5cb-405a-9f12-8c23f1c3238e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eae797065d574847b923a7ac71b8c804'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:32.607618 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-89118289-b5cb-405a-9f12-8c23f1c3238e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eae797065d574847b923a7ac71b8c804'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:32.620639 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-89118289-b5cb-405a-9f12-8c23f1c3238e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eae797065d574847b923a7ac71b8c804'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:32.621293 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-89118289-b5cb-405a-9f12-8c23f1c3238e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eae797065d574847b923a7ac71b8c804'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:32.621885 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-89118289-b5cb-405a-9f12-8c23f1c3238e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eae797065d574847b923a7ac71b8c804'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:32.634346 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1202/2423] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:59:32 2022] DELETE /identity/v3/users/eae797065d574847b923a7ac71b8c804 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:32.639002 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b22cb-69cb-4bed-b08a-5bd40c752fe9 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:32.639528 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b22cb-69cb-4bed-b08a-5bd40c752fe9 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:32.639528 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b22cb-69cb-4bed-b08a-5bd40c752fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:32.659277 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-7b2b22cb-69cb-4bed-b08a-5bd40c752fe9 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:32.660518 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-7b2b22cb-69cb-4bed-b08a-5bd40c752fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:32.695037 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:32.721044 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1222/2424] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:59:32.730587 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89118289-b5cb-405a-9f12-8c23f1c3238e admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:32.732012 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-89118289-b5cb-405a-9f12-8c23f1c3238e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:32.790722 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89118289-b5cb-405a-9f12-8c23f1c3238e admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:32.794487 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7829d8-73ba-4038-821f-780318eeeed9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:32.795084 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7829d8-73ba-4038-821f-780318eeeed9 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:32.795361 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7829d8-73ba-4038-821f-780318eeeed9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:32.795630 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7829d8-73ba-4038-821f-780318eeeed9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:32.805419 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7829d8-73ba-4038-821f-780318eeeed9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:32.806847 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7829d8-73ba-4038-821f-780318eeeed9 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:32.817031 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1203/2425] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:59:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:59:33.249120 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b22cb-69cb-4bed-b08a-5bd40c752fe9 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:33.250903 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-7b2b22cb-69cb-4bed-b08a-5bd40c752fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:33.264865 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b22cb-69cb-4bed-b08a-5bd40c752fe9 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:33.270830 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b131c-2371-4771-8753-9b03006a89c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:33.271476 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b131c-2371-4771-8753-9b03006a89c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:33.286042 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b131c-2371-4771-8753-9b03006a89c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:33.286421 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b131c-2371-4771-8753-9b03006a89c4 admin admin] PATH_INFO: `/v3/projects/eb26eaeda2e24f5fb7a80c2008f48209` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:33.287768 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b131c-2371-4771-8753-9b03006a89c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb26eaeda2e24f5fb7a80c2008f48209)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:33.289736 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b131c-2371-4771-8753-9b03006a89c4 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:33.385849 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-bb1b131c-2371-4771-8753-9b03006a89c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb26eaeda2e24f5fb7a80c2008f48209', 'request_id': 'req-bb1b131c-2371-4771-8753-9b03006a89c4'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:33.459073 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-bb1b131c-2371-4771-8753-9b03006a89c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb26eaeda2e24f5fb7a80c2008f48209', 'request_id': 'req-bb1b131c-2371-4771-8753-9b03006a89c4'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:33.465598 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-bb1b131c-2371-4771-8753-9b03006a89c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb26eaeda2e24f5fb7a80c2008f48209', 'request_id': 'req-bb1b131c-2371-4771-8753-9b03006a89c4'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:33.476962 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-bb1b131c-2371-4771-8753-9b03006a89c4 admin admin] The token cache is being invalidate because project eb26eaeda2e24f5fb7a80c2008f48209 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:59:33.477240 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-bb1b131c-2371-4771-8753-9b03006a89c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:33.477864 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-bb1b131c-2371-4771-8753-9b03006a89c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:33.479510 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1223/2426] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:59:33 2022] DELETE /identity/v3/projects/eb26eaeda2e24f5fb7a80c2008f48209 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:33.484036 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7829d8-73ba-4038-821f-780318eeeed9 service neutron] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:33.493961 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7829d8-73ba-4038-821f-780318eeeed9 service neutron] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:33.512021 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:33.550894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0eacc-2d04-412a-a025-69a1a957578e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:33.551452 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0eacc-2d04-412a-a025-69a1a957578e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:33.551756 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0eacc-2d04-412a-a025-69a1a957578e admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:33.552099 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0eacc-2d04-412a-a025-69a1a957578e admin admin] PATH_INFO: `/v3/users/b0d51c2125cf4f0ca1c60a8de3b727d9` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:33.553240 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0eacc-2d04-412a-a025-69a1a957578e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0d51c2125cf4f0ca1c60a8de3b727d9)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:33.555248 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0eacc-2d04-412a-a025-69a1a957578e admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:33.758766 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-24c0eacc-2d04-412a-a025-69a1a957578e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0d51c2125cf4f0ca1c60a8de3b727d9'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:33.835826 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-24c0eacc-2d04-412a-a025-69a1a957578e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0d51c2125cf4f0ca1c60a8de3b727d9'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:33.850993 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-24c0eacc-2d04-412a-a025-69a1a957578e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0d51c2125cf4f0ca1c60a8de3b727d9'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:33.851852 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-24c0eacc-2d04-412a-a025-69a1a957578e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0d51c2125cf4f0ca1c60a8de3b727d9'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:33.852393 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-24c0eacc-2d04-412a-a025-69a1a957578e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0d51c2125cf4f0ca1c60a8de3b727d9'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:33.864900 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1204/2427] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:59:33 2022] DELETE /identity/v3/users/b0d51c2125cf4f0ca1c60a8de3b727d9 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:34.271773 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b131c-2371-4771-8753-9b03006a89c4 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:34.274244 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-bb1b131c-2371-4771-8753-9b03006a89c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:34.303355 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b131c-2371-4771-8753-9b03006a89c4 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:34.369387 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847833ea-3a98-474a-b464-b96ee3867fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:34.370036 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847833ea-3a98-474a-b464-b96ee3867fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:34.370321 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847833ea-3a98-474a-b464-b96ee3867fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:34.370594 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847833ea-3a98-474a-b464-b96ee3867fd9 admin admin] PATH_INFO: `/v3/projects/3ec8abcd494a47008706ac384217b971` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:34.371761 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847833ea-3a98-474a-b464-b96ee3867fd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ec8abcd494a47008706ac384217b971)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:34.373676 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847833ea-3a98-474a-b464-b96ee3867fd9 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:34.471267 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-847833ea-3a98-474a-b464-b96ee3867fd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ec8abcd494a47008706ac384217b971', 'request_id': 'req-847833ea-3a98-474a-b464-b96ee3867fd9'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:34.550125 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-847833ea-3a98-474a-b464-b96ee3867fd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ec8abcd494a47008706ac384217b971', 'request_id': 'req-847833ea-3a98-474a-b464-b96ee3867fd9'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:34.556607 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-847833ea-3a98-474a-b464-b96ee3867fd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ec8abcd494a47008706ac384217b971', 'request_id': 'req-847833ea-3a98-474a-b464-b96ee3867fd9'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:34.558148 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:34.570071 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-847833ea-3a98-474a-b464-b96ee3867fd9 admin admin] The token cache is being invalidate because project 3ec8abcd494a47008706ac384217b971 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:59:34.570367 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-847833ea-3a98-474a-b464-b96ee3867fd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:34.571181 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-847833ea-3a98-474a-b464-b96ee3867fd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:34.573229 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1224/2428] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:59:34 2022] DELETE /identity/v3/projects/3ec8abcd494a47008706ac384217b971 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:34.577547 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0eacc-2d04-412a-a025-69a1a957578e admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:34.579722 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-24c0eacc-2d04-412a-a025-69a1a957578e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:34.594296 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0eacc-2d04-412a-a025-69a1a957578e admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:34.662518 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3dc4df-5d41-4331-92fa-0e37a0f09cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:34.663210 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3dc4df-5d41-4331-92fa-0e37a0f09cad admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:34.663494 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3dc4df-5d41-4331-92fa-0e37a0f09cad admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:34.663819 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3dc4df-5d41-4331-92fa-0e37a0f09cad admin admin] PATH_INFO: `/v3/users/461e2d6e0e7141df875674822fe02210` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:34.664968 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3dc4df-5d41-4331-92fa-0e37a0f09cad admin admin] RBAC: Authorizing `identity:delete_user(user_id=461e2d6e0e7141df875674822fe02210)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:34.666880 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3dc4df-5d41-4331-92fa-0e37a0f09cad admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:34.866757 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:34.869624 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-fc3dc4df-5d41-4331-92fa-0e37a0f09cad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '461e2d6e0e7141df875674822fe02210'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:34.977509 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-fc3dc4df-5d41-4331-92fa-0e37a0f09cad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '461e2d6e0e7141df875674822fe02210'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:34.990756 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-fc3dc4df-5d41-4331-92fa-0e37a0f09cad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '461e2d6e0e7141df875674822fe02210'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:34.991411 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-fc3dc4df-5d41-4331-92fa-0e37a0f09cad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '461e2d6e0e7141df875674822fe02210'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:34.992073 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-fc3dc4df-5d41-4331-92fa-0e37a0f09cad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '461e2d6e0e7141df875674822fe02210'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:35.005130 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1205/2429] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:59:34 2022] DELETE /identity/v3/users/461e2d6e0e7141df875674822fe02210 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:35.449765 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847833ea-3a98-474a-b464-b96ee3867fd9 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:35.449765 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-847833ea-3a98-474a-b464-b96ee3867fd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:35.464007 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847833ea-3a98-474a-b464-b96ee3867fd9 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:35.522838 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afae32d4-95f3-4eda-80a9-63c0c6dc05ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:35.523463 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae32d4-95f3-4eda-80a9-63c0c6dc05ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:35.523764 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae32d4-95f3-4eda-80a9-63c0c6dc05ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:35.524035 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afae32d4-95f3-4eda-80a9-63c0c6dc05ef admin admin] PATH_INFO: `/v3/projects/ae4b98ea4f204ead8649e08ea59fa03b` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:35.525140 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afae32d4-95f3-4eda-80a9-63c0c6dc05ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae4b98ea4f204ead8649e08ea59fa03b)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:35.527042 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afae32d4-95f3-4eda-80a9-63c0c6dc05ef admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:35.620753 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-afae32d4-95f3-4eda-80a9-63c0c6dc05ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae4b98ea4f204ead8649e08ea59fa03b', 'request_id': 'req-afae32d4-95f3-4eda-80a9-63c0c6dc05ef'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:35.691397 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-afae32d4-95f3-4eda-80a9-63c0c6dc05ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae4b98ea4f204ead8649e08ea59fa03b', 'request_id': 'req-afae32d4-95f3-4eda-80a9-63c0c6dc05ef'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:35.697572 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-afae32d4-95f3-4eda-80a9-63c0c6dc05ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae4b98ea4f204ead8649e08ea59fa03b', 'request_id': 'req-afae32d4-95f3-4eda-80a9-63c0c6dc05ef'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:35.711688 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-afae32d4-95f3-4eda-80a9-63c0c6dc05ef admin admin] The token cache is being invalidate because project ae4b98ea4f204ead8649e08ea59fa03b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:59:35.711688 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-afae32d4-95f3-4eda-80a9-63c0c6dc05ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:35.711688 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-afae32d4-95f3-4eda-80a9-63c0c6dc05ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:35.712052 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1225/2430] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:59:35 2022] DELETE /identity/v3/projects/ae4b98ea4f204ead8649e08ea59fa03b => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:35.717035 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59648d81-2abf-46ff-b951-9dd248dc0de5 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:35.717521 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59648d81-2abf-46ff-b951-9dd248dc0de5 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:35.717623 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59648d81-2abf-46ff-b951-9dd248dc0de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:35.738910 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-59648d81-2abf-46ff-b951-9dd248dc0de5 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:35.740601 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-59648d81-2abf-46ff-b951-9dd248dc0de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:35.799114 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1206/2431] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:59:35.862509 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fb5e7-72f0-4b4d-ad53-1bb822cd7845 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:35.862786 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fb5e7-72f0-4b4d-ad53-1bb822cd7845 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:35.863057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55fb5e7-72f0-4b4d-ad53-1bb822cd7845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:35.884308 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-e55fb5e7-72f0-4b4d-ad53-1bb822cd7845 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:35.885518 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-e55fb5e7-72f0-4b4d-ad53-1bb822cd7845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:35.893399 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1226/2432] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:59:35.897701 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59648d81-2abf-46ff-b951-9dd248dc0de5 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:35.907897 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59648d81-2abf-46ff-b951-9dd248dc0de5 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:35.912570 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f60009-0d05-4843-8abb-0ce2bab535f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:35.913168 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f60009-0d05-4843-8abb-0ce2bab535f1 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:35.913449 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f60009-0d05-4843-8abb-0ce2bab535f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:35.913722 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f60009-0d05-4843-8abb-0ce2bab535f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:35.914368 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f60009-0d05-4843-8abb-0ce2bab535f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:59:35.914674 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f60009-0d05-4843-8abb-0ce2bab535f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:35.916277 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f60009-0d05-4843-8abb-0ce2bab535f1 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:35.928445 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1207/2433] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:59:35.933289 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55fb5e7-72f0-4b4d-ad53-1bb822cd7845 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:35.943543 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55fb5e7-72f0-4b4d-ad53-1bb822cd7845 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:35.948170 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4387be52-853b-4992-b940-79e3d6903c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:35.948772 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387be52-853b-4992-b940-79e3d6903c2c admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:35.949046 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387be52-853b-4992-b940-79e3d6903c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:35.949313 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4387be52-853b-4992-b940-79e3d6903c2c admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:35.949958 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4387be52-853b-4992-b940-79e3d6903c2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:35.951892 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4387be52-853b-4992-b940-79e3d6903c2c admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:36.036389 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1227/2434] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:59:35 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:59:36.040586 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f60009-0d05-4843-8abb-0ce2bab535f1 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:36.051560 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f60009-0d05-4843-8abb-0ce2bab535f1 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:36.122033 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d1c87-c152-4b6d-9038-c2f648d49751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:36.123729 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d1c87-c152-4b6d-9038-c2f648d49751 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:36.123729 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d1c87-c152-4b6d-9038-c2f648d49751 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:36.123729 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d1c87-c152-4b6d-9038-c2f648d49751 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:36.124120 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d1c87-c152-4b6d-9038-c2f648d49751 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:36.126217 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1d1c87-c152-4b6d-9038-c2f648d49751 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:36.241679 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1208/2435] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:59:36 2022] POST /identity/v3/users => generated 286 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:59:36.246090 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4387be52-853b-4992-b940-79e3d6903c2c admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:36.256936 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4387be52-853b-4992-b940-79e3d6903c2c admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:36.261583 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a1636-4686-407a-bc83-3db1b51766b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:36.262188 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a1636-4686-407a-bc83-3db1b51766b5 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:36.262467 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a1636-4686-407a-bc83-3db1b51766b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:36.262742 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a1636-4686-407a-bc83-3db1b51766b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:36.263441 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a1636-4686-407a-bc83-3db1b51766b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:36.265059 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0a1636-4686-407a-bc83-3db1b51766b5 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:36.276045 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1228/2436] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:59:36.280280 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d1c87-c152-4b6d-9038-c2f648d49751 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:36.290871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d1c87-c152-4b6d-9038-c2f648d49751 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:36.295544 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb60db9-78e6-478c-b61c-40c183e92108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:36.296226 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb60db9-78e6-478c-b61c-40c183e92108 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:36.296518 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb60db9-78e6-478c-b61c-40c183e92108 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:36.296786 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb60db9-78e6-478c-b61c-40c183e92108 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:36.297448 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb60db9-78e6-478c-b61c-40c183e92108 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:36.299351 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb60db9-78e6-478c-b61c-40c183e92108 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:36.312539 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1209/2437] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:59:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:59:36.316951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a1636-4686-407a-bc83-3db1b51766b5 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:36.327460 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0a1636-4686-407a-bc83-3db1b51766b5 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:36.332750 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac0811-2554-440f-aa61-fb2cb95f3b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:36.333411 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac0811-2554-440f-aa61-fb2cb95f3b2b admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:36.333711 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac0811-2554-440f-aa61-fb2cb95f3b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:36.333982 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ac0811-2554-440f-aa61-fb2cb95f3b2b admin admin] PATH_INFO: `/v3/projects/87537590ed3b449b91be75cf1811e527/users/07aaabfabeec47938c3516339682c046/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:36.350066 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ac0811-2554-440f-aa61-fb2cb95f3b2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=87537590ed3b449b91be75cf1811e527, user_id=07aaabfabeec47938c3516339682c046, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:36.352760 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ac0811-2554-440f-aa61-fb2cb95f3b2b admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:36.452157 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1229/2438] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:59:36 2022] PUT /identity/v3/projects/87537590ed3b449b91be75cf1811e527/users/07aaabfabeec47938c3516339682c046/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:36.462887 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e52956-4837-4e59-998e-ae20765ec229 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:36.463177 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e52956-4837-4e59-998e-ae20765ec229 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:36.463456 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e52956-4837-4e59-998e-ae20765ec229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:36.490865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-88e52956-4837-4e59-998e-ae20765ec229 None None] MFA Rules not processed for user `07aaabfabeec47938c3516339682c046`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:36.492109 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-88e52956-4837-4e59-998e-ae20765ec229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:36.504826 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:36.557870 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1210/2439] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:59:36.569828 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac0811-2554-440f-aa61-fb2cb95f3b2b admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:36.571554 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-a5ac0811-2554-440f-aa61-fb2cb95f3b2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:36.594625 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:36.630970 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ac0811-2554-440f-aa61-fb2cb95f3b2b admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:36.634622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1b2f6a-3760-4b11-9b15-b49ce42ce65a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '565a836c7f89478cad615bd6c14531a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:36.635231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b2f6a-3760-4b11-9b15-b49ce42ce65a service glance] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:36.635511 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b2f6a-3760-4b11-9b15-b49ce42ce65a service glance] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:36.635816 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1b2f6a-3760-4b11-9b15-b49ce42ce65a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:36.644661 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b2f6a-3760-4b11-9b15-b49ce42ce65a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:36.646089 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1b2f6a-3760-4b11-9b15-b49ce42ce65a service glance] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:36.654717 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1230/2440] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:59:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:59:37.409750 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e52956-4837-4e59-998e-ae20765ec229 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:37.411336 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-88e52956-4837-4e59-998e-ae20765ec229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:37.479516 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e52956-4837-4e59-998e-ae20765ec229 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:37.487025 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29743fab-5d86-4ac1-be18-f2811e2bccc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:37.487025 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29743fab-5d86-4ac1-be18-f2811e2bccc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:37.487025 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29743fab-5d86-4ac1-be18-f2811e2bccc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:37.487025 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29743fab-5d86-4ac1-be18-f2811e2bccc2 admin admin] PATH_INFO: `/v3/users/6ee922928e1f467a86f7ea3f1736d20d` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:37.487633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29743fab-5d86-4ac1-be18-f2811e2bccc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ee922928e1f467a86f7ea3f1736d20d)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:37.489563 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29743fab-5d86-4ac1-be18-f2811e2bccc2 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:37.661759 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-29743fab-5d86-4ac1-be18-f2811e2bccc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ee922928e1f467a86f7ea3f1736d20d'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:37.721590 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-29743fab-5d86-4ac1-be18-f2811e2bccc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ee922928e1f467a86f7ea3f1736d20d'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:37.734221 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-29743fab-5d86-4ac1-be18-f2811e2bccc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ee922928e1f467a86f7ea3f1736d20d'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:37.734860 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-29743fab-5d86-4ac1-be18-f2811e2bccc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ee922928e1f467a86f7ea3f1736d20d'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:37.735453 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-29743fab-5d86-4ac1-be18-f2811e2bccc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ee922928e1f467a86f7ea3f1736d20d'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:37.747692 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1211/2441] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:59:37 2022] DELETE /identity/v3/users/6ee922928e1f467a86f7ea3f1736d20d => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:59:37.752265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ddb0ef-b201-4fd9-a7e5-3f9a52e68ce8 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:37.752720 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ddb0ef-b201-4fd9-a7e5-3f9a52e68ce8 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:37.752987 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ddb0ef-b201-4fd9-a7e5-3f9a52e68ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:37.772295 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-f2ddb0ef-b201-4fd9-a7e5-3f9a52e68ce8 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:37.773489 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-f2ddb0ef-b201-4fd9-a7e5-3f9a52e68ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:37.832746 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1231/2442] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:59:37.844027 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29743fab-5d86-4ac1-be18-f2811e2bccc2 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:37.845431 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-29743fab-5d86-4ac1-be18-f2811e2bccc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:37.886223 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:37.903962 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29743fab-5d86-4ac1-be18-f2811e2bccc2 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:37.908441 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e2c9d-5158-4617-8725-95903d7e3239 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:37.909103 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e2c9d-5158-4617-8725-95903d7e3239 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:37.909386 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e2c9d-5158-4617-8725-95903d7e3239 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:37.909659 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e2c9d-5158-4617-8725-95903d7e3239 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:37.919132 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e2c9d-5158-4617-8725-95903d7e3239 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:37.920731 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e2c9d-5158-4617-8725-95903d7e3239 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:37.931360 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1212/2443] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:59:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:59:38.312434 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ddb0ef-b201-4fd9-a7e5-3f9a52e68ce8 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:38.313867 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-f2ddb0ef-b201-4fd9-a7e5-3f9a52e68ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:38.325997 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ddb0ef-b201-4fd9-a7e5-3f9a52e68ce8 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:38.331483 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac23257-bbd9-4028-a824-58f15a07ff48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:38.332165 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac23257-bbd9-4028-a824-58f15a07ff48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:38.332480 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac23257-bbd9-4028-a824-58f15a07ff48 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:38.333039 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac23257-bbd9-4028-a824-58f15a07ff48 admin admin] PATH_INFO: `/v3/projects/ede6946540ee486cbee59ecc18817fcd` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:38.333910 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac23257-bbd9-4028-a824-58f15a07ff48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ede6946540ee486cbee59ecc18817fcd)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:38.335840 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac23257-bbd9-4028-a824-58f15a07ff48 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:38.419295 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-8ac23257-bbd9-4028-a824-58f15a07ff48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ede6946540ee486cbee59ecc18817fcd', 'request_id': 'req-8ac23257-bbd9-4028-a824-58f15a07ff48'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:38.473804 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-8ac23257-bbd9-4028-a824-58f15a07ff48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ede6946540ee486cbee59ecc18817fcd', 'request_id': 'req-8ac23257-bbd9-4028-a824-58f15a07ff48'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:38.485542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-8ac23257-bbd9-4028-a824-58f15a07ff48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ede6946540ee486cbee59ecc18817fcd', 'request_id': 'req-8ac23257-bbd9-4028-a824-58f15a07ff48'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:38.492019 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-8ac23257-bbd9-4028-a824-58f15a07ff48 admin admin] The token cache is being invalidate because project ede6946540ee486cbee59ecc18817fcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:59:38.492439 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-8ac23257-bbd9-4028-a824-58f15a07ff48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:38.493209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-8ac23257-bbd9-4028-a824-58f15a07ff48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:38.494761 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1232/2444] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:59:38 2022] DELETE /identity/v3/projects/ede6946540ee486cbee59ecc18817fcd => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:38.498852 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e2c9d-5158-4617-8725-95903d7e3239 service neutron] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:38.512518 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e2c9d-5158-4617-8725-95903d7e3239 service neutron] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:38.564570 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c3468c-25ed-4f36-8e7c-ab760ba0b169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:38.569631 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3468c-25ed-4f36-8e7c-ab760ba0b169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:38.569631 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3468c-25ed-4f36-8e7c-ab760ba0b169 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:38.569631 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c3468c-25ed-4f36-8e7c-ab760ba0b169 admin admin] PATH_INFO: `/v3/users/2c470718caa44deca6f100a2758c3cfc` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:38.569631 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c3468c-25ed-4f36-8e7c-ab760ba0b169 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c470718caa44deca6f100a2758c3cfc)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:38.569631 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c3468c-25ed-4f36-8e7c-ab760ba0b169 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:38.768672 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-e0c3468c-25ed-4f36-8e7c-ab760ba0b169 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c470718caa44deca6f100a2758c3cfc'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:38.840073 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-e0c3468c-25ed-4f36-8e7c-ab760ba0b169 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c470718caa44deca6f100a2758c3cfc'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:38.852912 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-e0c3468c-25ed-4f36-8e7c-ab760ba0b169 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c470718caa44deca6f100a2758c3cfc'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:38.856097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-e0c3468c-25ed-4f36-8e7c-ab760ba0b169 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c470718caa44deca6f100a2758c3cfc'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:38.856097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-e0c3468c-25ed-4f36-8e7c-ab760ba0b169 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c470718caa44deca6f100a2758c3cfc'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:38.866205 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1213/2445] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:59:38 2022] DELETE /identity/v3/users/2c470718caa44deca6f100a2758c3cfc => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:39.289015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac23257-bbd9-4028-a824-58f15a07ff48 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:39.290445 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-8ac23257-bbd9-4028-a824-58f15a07ff48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:39.304103 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac23257-bbd9-4028-a824-58f15a07ff48 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:39.350983 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:39.363571 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3025b065-387d-4163-8cb9-5d420bcd47dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:39.364267 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025b065-387d-4163-8cb9-5d420bcd47dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:39.364548 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025b065-387d-4163-8cb9-5d420bcd47dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:39.364819 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025b065-387d-4163-8cb9-5d420bcd47dc admin admin] PATH_INFO: `/v3/projects/ddcd21a4dda14f489088dd2be9982a07` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:39.366030 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3025b065-387d-4163-8cb9-5d420bcd47dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddcd21a4dda14f489088dd2be9982a07)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:39.367978 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3025b065-387d-4163-8cb9-5d420bcd47dc admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:39.475853 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3025b065-387d-4163-8cb9-5d420bcd47dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddcd21a4dda14f489088dd2be9982a07', 'request_id': 'req-3025b065-387d-4163-8cb9-5d420bcd47dc'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:39.542650 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3025b065-387d-4163-8cb9-5d420bcd47dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddcd21a4dda14f489088dd2be9982a07', 'request_id': 'req-3025b065-387d-4163-8cb9-5d420bcd47dc'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:39.549413 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3025b065-387d-4163-8cb9-5d420bcd47dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddcd21a4dda14f489088dd2be9982a07', 'request_id': 'req-3025b065-387d-4163-8cb9-5d420bcd47dc'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:39.560758 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3025b065-387d-4163-8cb9-5d420bcd47dc admin admin] The token cache is being invalidate because project ddcd21a4dda14f489088dd2be9982a07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:59:39.561047 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3025b065-387d-4163-8cb9-5d420bcd47dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:39.561683 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-3025b065-387d-4163-8cb9-5d420bcd47dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:39.563241 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1233/2446] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:59:39 2022] DELETE /identity/v3/projects/ddcd21a4dda14f489088dd2be9982a07 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:39.568183 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8fee3-6ded-41f8-aae7-7378276a0ae4 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:39.569323 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8fee3-6ded-41f8-aae7-7378276a0ae4 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:39.569479 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8fee3-6ded-41f8-aae7-7378276a0ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:39.589570 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-4fe8fee3-6ded-41f8-aae7-7378276a0ae4 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:39.590803 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-4fe8fee3-6ded-41f8-aae7-7378276a0ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:39.645514 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:39.654746 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1214/2447] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:59:39.662997 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea6ff9-761e-46c0-897b-df0b6d64302a None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:39.663282 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea6ff9-761e-46c0-897b-df0b6d64302a None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:39.663555 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feea6ff9-761e-46c0-897b-df0b6d64302a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:39.685005 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-feea6ff9-761e-46c0-897b-df0b6d64302a None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:39.686220 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-feea6ff9-761e-46c0-897b-df0b6d64302a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:39.694282 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1234/2448] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:59:39.698527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8fee3-6ded-41f8-aae7-7378276a0ae4 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:39.708597 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8fee3-6ded-41f8-aae7-7378276a0ae4 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:39.713464 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666674da-1011-4562-a251-721dabb5144d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:39.714350 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666674da-1011-4562-a251-721dabb5144d admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:39.714690 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666674da-1011-4562-a251-721dabb5144d admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:39.715044 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666674da-1011-4562-a251-721dabb5144d admin admin] PATH_INFO: `/v3/domains` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:39.715532 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666674da-1011-4562-a251-721dabb5144d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88300) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:59:39.715863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666674da-1011-4562-a251-721dabb5144d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:39.717442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666674da-1011-4562-a251-721dabb5144d admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:39.730266 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1215/2449] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:59:39.734991 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea6ff9-761e-46c0-897b-df0b6d64302a None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:39.744909 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feea6ff9-761e-46c0-897b-df0b6d64302a None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:39.749561 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ef1ca-a3be-4b83-af01-60fb878f904b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:39.750156 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ef1ca-a3be-4b83-af01-60fb878f904b admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:39.750433 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ef1ca-a3be-4b83-af01-60fb878f904b admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:39.750703 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ef1ca-a3be-4b83-af01-60fb878f904b admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:39.751341 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ef1ca-a3be-4b83-af01-60fb878f904b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:39.753263 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ef1ca-a3be-4b83-af01-60fb878f904b admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:39.850464 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1235/2450] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:59:39 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:59:39.854319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666674da-1011-4562-a251-721dabb5144d admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:39.864660 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666674da-1011-4562-a251-721dabb5144d admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:39.920191 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0baf97-cb27-4071-9cd6-de16a714a936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:39.920809 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0baf97-cb27-4071-9cd6-de16a714a936 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:39.921090 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0baf97-cb27-4071-9cd6-de16a714a936 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:39.921386 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0baf97-cb27-4071-9cd6-de16a714a936 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:39.922038 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0baf97-cb27-4071-9cd6-de16a714a936 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:39.923996 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0baf97-cb27-4071-9cd6-de16a714a936 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:40.013862 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1216/2451] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:59:39 2022] POST /identity/v3/users => generated 291 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:59:40.018425 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ef1ca-a3be-4b83-af01-60fb878f904b admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:40.028590 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ef1ca-a3be-4b83-af01-60fb878f904b admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:40.033739 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e6feb-5e9a-4815-b2a3-5dc5d2129a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:40.034327 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e6feb-5e9a-4815-b2a3-5dc5d2129a98 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:40.034601 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e6feb-5e9a-4815-b2a3-5dc5d2129a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:40.034869 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2e6feb-5e9a-4815-b2a3-5dc5d2129a98 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:40.035499 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2e6feb-5e9a-4815-b2a3-5dc5d2129a98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:40.037101 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2e6feb-5e9a-4815-b2a3-5dc5d2129a98 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:40.047591 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1236/2452] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:59:40.051623 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0baf97-cb27-4071-9cd6-de16a714a936 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:40.061641 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0baf97-cb27-4071-9cd6-de16a714a936 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:40.066227 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25996420-31d9-4a77-9ec2-ba28fcf97515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:40.066820 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25996420-31d9-4a77-9ec2-ba28fcf97515 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:40.067105 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25996420-31d9-4a77-9ec2-ba28fcf97515 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:40.067381 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25996420-31d9-4a77-9ec2-ba28fcf97515 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:40.068063 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25996420-31d9-4a77-9ec2-ba28fcf97515 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:40.069643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25996420-31d9-4a77-9ec2-ba28fcf97515 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:40.080588 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1217/2453] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:59:40.084461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e6feb-5e9a-4815-b2a3-5dc5d2129a98 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:40.094528 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2e6feb-5e9a-4815-b2a3-5dc5d2129a98 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:40.099846 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0f2b0-bb25-4ecf-b41b-5c3dd20e2aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:40.100629 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0f2b0-bb25-4ecf-b41b-5c3dd20e2aeb admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:40.100800 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0f2b0-bb25-4ecf-b41b-5c3dd20e2aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:40.101011 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae0f2b0-bb25-4ecf-b41b-5c3dd20e2aeb admin admin] PATH_INFO: `/v3/projects/b4525f2e4ce64369be50bfcbbc81bed5/users/a67f11786d614abe8e3a554c91495797/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:40.116784 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae0f2b0-bb25-4ecf-b41b-5c3dd20e2aeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4525f2e4ce64369be50bfcbbc81bed5, user_id=a67f11786d614abe8e3a554c91495797, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:40.119437 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae0f2b0-bb25-4ecf-b41b-5c3dd20e2aeb admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:40.185238 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1237/2454] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:59:40 2022] PUT /identity/v3/projects/b4525f2e4ce64369be50bfcbbc81bed5/users/a67f11786d614abe8e3a554c91495797/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:40.250451 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5cbfc-a73c-492b-b673-e7e93b498da4 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:40.250738 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5cbfc-a73c-492b-b673-e7e93b498da4 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:40.251032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5cbfc-a73c-492b-b673-e7e93b498da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:40.278998 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-eae5cbfc-a73c-492b-b673-e7e93b498da4 None None] MFA Rules not processed for user `a67f11786d614abe8e3a554c91495797`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:40.280232 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-eae5cbfc-a73c-492b-b673-e7e93b498da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:40.345937 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1218/2455] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:59:40.350476 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0f2b0-bb25-4ecf-b41b-5c3dd20e2aeb admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:40.412557 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae0f2b0-bb25-4ecf-b41b-5c3dd20e2aeb admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:40.417208 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e3656-8bef-4b78-a60d-0b5916a822ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:40.417807 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e3656-8bef-4b78-a60d-0b5916a822ee admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:40.421875 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e3656-8bef-4b78-a60d-0b5916a822ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:40.421875 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e3656-8bef-4b78-a60d-0b5916a822ee admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:40.421875 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e3656-8bef-4b78-a60d-0b5916a822ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:40.421875 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e3656-8bef-4b78-a60d-0b5916a822ee admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:40.505205 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:40.512333 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1238/2456] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:59:40 2022] POST /identity/v3/projects => generated 379 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:59:40.516379 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5cbfc-a73c-492b-b673-e7e93b498da4 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:40.527266 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5cbfc-a73c-492b-b673-e7e93b498da4 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:40.575834 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:40.585213 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3093de-053e-4d2d-b64e-4fcdb83db41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:40.585856 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3093de-053e-4d2d-b64e-4fcdb83db41f admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:40.586119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3093de-053e-4d2d-b64e-4fcdb83db41f admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:40.586422 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3093de-053e-4d2d-b64e-4fcdb83db41f admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:40.587068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3093de-053e-4d2d-b64e-4fcdb83db41f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:40.589286 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3093de-053e-4d2d-b64e-4fcdb83db41f admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:40.712526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1219/2457] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:59:40 2022] POST /identity/v3/users => generated 296 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:59:40.717027 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e3656-8bef-4b78-a60d-0b5916a822ee admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:40.727096 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e3656-8bef-4b78-a60d-0b5916a822ee admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:40.731550 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094bb87-b331-4f4b-911c-7e529fae8c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:40.732187 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094bb87-b331-4f4b-911c-7e529fae8c2a admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:40.732469 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094bb87-b331-4f4b-911c-7e529fae8c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:40.732741 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094bb87-b331-4f4b-911c-7e529fae8c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:40.733373 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b094bb87-b331-4f4b-911c-7e529fae8c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:40.734960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b094bb87-b331-4f4b-911c-7e529fae8c2a admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:40.745717 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1239/2458] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:59:40.749728 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3093de-053e-4d2d-b64e-4fcdb83db41f admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:40.760646 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3093de-053e-4d2d-b64e-4fcdb83db41f admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:40.765271 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7e15a-f400-4b6f-bd97-9bd496fad42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:40.766019 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7e15a-f400-4b6f-bd97-9bd496fad42e admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:40.766592 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7e15a-f400-4b6f-bd97-9bd496fad42e admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:40.766888 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f7e15a-f400-4b6f-bd97-9bd496fad42e admin admin] PATH_INFO: `/v3/projects/39b00bcf12ec4a6fa2a4f064f245fb3c/users/8f6f0fe1e1844ee28e0338383c091ed0/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:40.781737 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7e15a-f400-4b6f-bd97-9bd496fad42e admin admin] RBAC: Authorizing `identity:create_grant(project_id=39b00bcf12ec4a6fa2a4f064f245fb3c, user_id=8f6f0fe1e1844ee28e0338383c091ed0, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:40.784412 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f7e15a-f400-4b6f-bd97-9bd496fad42e admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:40.853804 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1220/2459] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:59:40 2022] PUT /identity/v3/projects/39b00bcf12ec4a6fa2a4f064f245fb3c/users/8f6f0fe1e1844ee28e0338383c091ed0/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:40.858549 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094bb87-b331-4f4b-911c-7e529fae8c2a admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:40.870987 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094bb87-b331-4f4b-911c-7e529fae8c2a admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:40.925844 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa74527-6d37-4d23-8f75-3c87745ad0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:40.926445 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa74527-6d37-4d23-8f75-3c87745ad0b0 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:40.926755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa74527-6d37-4d23-8f75-3c87745ad0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:40.927029 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa74527-6d37-4d23-8f75-3c87745ad0b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:40.927659 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa74527-6d37-4d23-8f75-3c87745ad0b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:40.929266 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa74527-6d37-4d23-8f75-3c87745ad0b0 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:40.939512 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1240/2460] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:59:40 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:59:40.943705 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7e15a-f400-4b6f-bd97-9bd496fad42e admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:40.953983 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f7e15a-f400-4b6f-bd97-9bd496fad42e admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:40.958812 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc3132-a0b6-4a21-b1db-cdf2bd08b824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:40.959470 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc3132-a0b6-4a21-b1db-cdf2bd08b824 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:40.959730 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc3132-a0b6-4a21-b1db-cdf2bd08b824 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:40.960032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc3132-a0b6-4a21-b1db-cdf2bd08b824 admin admin] PATH_INFO: `/v3/projects/39b00bcf12ec4a6fa2a4f064f245fb3c/users/8f6f0fe1e1844ee28e0338383c091ed0/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:40.961912 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc3132-a0b6-4a21-b1db-cdf2bd08b824 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39b00bcf12ec4a6fa2a4f064f245fb3c, user_id=8f6f0fe1e1844ee28e0338383c091ed0, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:40.964804 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc3132-a0b6-4a21-b1db-cdf2bd08b824 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:40.975173 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1221/2461] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:59:40 2022] PUT /identity/v3/projects/39b00bcf12ec4a6fa2a4f064f245fb3c/users/8f6f0fe1e1844ee28e0338383c091ed0/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:40.984963 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e6492-4a81-431a-8092-091e7775aec9 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:40.985256 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e6492-4a81-431a-8092-091e7775aec9 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:40.985540 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e6492-4a81-431a-8092-091e7775aec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:41.014480 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-510e6492-4a81-431a-8092-091e7775aec9 None None] MFA Rules not processed for user `8f6f0fe1e1844ee28e0338383c091ed0`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:41.015695 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-510e6492-4a81-431a-8092-091e7775aec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:41.083640 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1241/2462] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:40 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:59:41.095717 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc3132-a0b6-4a21-b1db-cdf2bd08b824 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:41.098340 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-a7fc3132-a0b6-4a21-b1db-cdf2bd08b824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:41.156673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc3132-a0b6-4a21-b1db-cdf2bd08b824 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:41.160884 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d879f004-b93a-45ba-8b4e-c7e967e09224 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:41.161496 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879f004-b93a-45ba-8b4e-c7e967e09224 service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:41.161774 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879f004-b93a-45ba-8b4e-c7e967e09224 service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:41.162046 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879f004-b93a-45ba-8b4e-c7e967e09224 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:41.171240 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d879f004-b93a-45ba-8b4e-c7e967e09224 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:41.172679 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d879f004-b93a-45ba-8b4e-c7e967e09224 service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:41.182453 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1222/2463] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:59:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:59:41.697409 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e6492-4a81-431a-8092-091e7775aec9 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:41.764887 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e6492-4a81-431a-8092-091e7775aec9 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:41.769326 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f4c27-3e02-42e2-bf3e-4f4d6b0ea942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:41.769953 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f4c27-3e02-42e2-bf3e-4f4d6b0ea942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:41.770230 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f4c27-3e02-42e2-bf3e-4f4d6b0ea942 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:41.770500 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f4c27-3e02-42e2-bf3e-4f4d6b0ea942 admin admin] PATH_INFO: `/v3/users/a67f11786d614abe8e3a554c91495797` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:41.771602 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5f4c27-3e02-42e2-bf3e-4f4d6b0ea942 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a67f11786d614abe8e3a554c91495797)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:41.773529 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5f4c27-3e02-42e2-bf3e-4f4d6b0ea942 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:41.783910 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:41.976170 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-fb5f4c27-3e02-42e2-bf3e-4f4d6b0ea942 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a67f11786d614abe8e3a554c91495797'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:41.976816 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-fb5f4c27-3e02-42e2-bf3e-4f4d6b0ea942 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a67f11786d614abe8e3a554c91495797'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:42.064912 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-fb5f4c27-3e02-42e2-bf3e-4f4d6b0ea942 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a67f11786d614abe8e3a554c91495797'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:42.077879 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-fb5f4c27-3e02-42e2-bf3e-4f4d6b0ea942 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a67f11786d614abe8e3a554c91495797'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:42.078584 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-fb5f4c27-3e02-42e2-bf3e-4f4d6b0ea942 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a67f11786d614abe8e3a554c91495797'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:42.091456 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1242/2464] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:59:41 2022] DELETE /identity/v3/users/a67f11786d614abe8e3a554c91495797 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:42.096076 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1920c8f-4bef-4b82-a595-040872f18363 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:42.096574 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1920c8f-4bef-4b82-a595-040872f18363 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:42.096675 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1920c8f-4bef-4b82-a595-040872f18363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:42.116302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-f1920c8f-4bef-4b82-a595-040872f18363 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:42.117488 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-f1920c8f-4bef-4b82-a595-040872f18363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:42.182470 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1223/2465] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:59:42.189697 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f4c27-3e02-42e2-bf3e-4f4d6b0ea942 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:42.191129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-fb5f4c27-3e02-42e2-bf3e-4f4d6b0ea942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:42.248124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f4c27-3e02-42e2-bf3e-4f4d6b0ea942 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:42.251822 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d6928-1566-4cbc-8b86-066e5d2ec486 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:42.252424 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d6928-1566-4cbc-8b86-066e5d2ec486 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:42.252699 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d6928-1566-4cbc-8b86-066e5d2ec486 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:42.252968 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d6928-1566-4cbc-8b86-066e5d2ec486 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:42.264543 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d6928-1566-4cbc-8b86-066e5d2ec486 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:42.266004 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d6928-1566-4cbc-8b86-066e5d2ec486 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:42.276411 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1243/2466] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:59:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:59:42.731878 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1920c8f-4bef-4b82-a595-040872f18363 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:42.733247 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-f1920c8f-4bef-4b82-a595-040872f18363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:42.751950 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1920c8f-4bef-4b82-a595-040872f18363 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:42.754403 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5daf85-9299-4dd7-8777-ab08f48644e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:42.755219 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5daf85-9299-4dd7-8777-ab08f48644e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:42.755731 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5daf85-9299-4dd7-8777-ab08f48644e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:42.755959 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5daf85-9299-4dd7-8777-ab08f48644e8 admin admin] PATH_INFO: `/v3/projects/b4525f2e4ce64369be50bfcbbc81bed5` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:42.757056 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5daf85-9299-4dd7-8777-ab08f48644e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4525f2e4ce64369be50bfcbbc81bed5)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:42.759041 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5daf85-9299-4dd7-8777-ab08f48644e8 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:42.989302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:42.993076 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-dd5daf85-9299-4dd7-8777-ab08f48644e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4525f2e4ce64369be50bfcbbc81bed5', 'request_id': 'req-dd5daf85-9299-4dd7-8777-ab08f48644e8'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:43.081793 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-dd5daf85-9299-4dd7-8777-ab08f48644e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4525f2e4ce64369be50bfcbbc81bed5', 'request_id': 'req-dd5daf85-9299-4dd7-8777-ab08f48644e8'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:43.088001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-dd5daf85-9299-4dd7-8777-ab08f48644e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4525f2e4ce64369be50bfcbbc81bed5', 'request_id': 'req-dd5daf85-9299-4dd7-8777-ab08f48644e8'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:43.099861 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-dd5daf85-9299-4dd7-8777-ab08f48644e8 admin admin] The token cache is being invalidate because project b4525f2e4ce64369be50bfcbbc81bed5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:59:43.100537 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-dd5daf85-9299-4dd7-8777-ab08f48644e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:43.100857 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-dd5daf85-9299-4dd7-8777-ab08f48644e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:43.102345 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1224/2467] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:59:42 2022] DELETE /identity/v3/projects/b4525f2e4ce64369be50bfcbbc81bed5 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:43.106316 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d6928-1566-4cbc-8b86-066e5d2ec486 service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:43.116704 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d6928-1566-4cbc-8b86-066e5d2ec486 service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:43.171269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9eafc7-e176-45bc-9949-30cbf2f86ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:43.171923 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9eafc7-e176-45bc-9949-30cbf2f86ac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:43.172203 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9eafc7-e176-45bc-9949-30cbf2f86ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:43.172473 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9eafc7-e176-45bc-9949-30cbf2f86ac1 admin admin] PATH_INFO: `/v3/users/8f6f0fe1e1844ee28e0338383c091ed0` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:43.173671 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9eafc7-e176-45bc-9949-30cbf2f86ac1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f6f0fe1e1844ee28e0338383c091ed0)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:43.175557 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9eafc7-e176-45bc-9949-30cbf2f86ac1 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:43.361994 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:43.374201 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-7d9eafc7-e176-45bc-9949-30cbf2f86ac1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f6f0fe1e1844ee28e0338383c091ed0'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:43.374835 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-7d9eafc7-e176-45bc-9949-30cbf2f86ac1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f6f0fe1e1844ee28e0338383c091ed0'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:43.447545 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-7d9eafc7-e176-45bc-9949-30cbf2f86ac1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f6f0fe1e1844ee28e0338383c091ed0'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:43.459880 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-7d9eafc7-e176-45bc-9949-30cbf2f86ac1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f6f0fe1e1844ee28e0338383c091ed0'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:43.460508 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-7d9eafc7-e176-45bc-9949-30cbf2f86ac1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f6f0fe1e1844ee28e0338383c091ed0'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:43.472622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1244/2468] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:59:43 2022] DELETE /identity/v3/users/8f6f0fe1e1844ee28e0338383c091ed0 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:43.889132 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5daf85-9299-4dd7-8777-ab08f48644e8 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:43.890673 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-dd5daf85-9299-4dd7-8777-ab08f48644e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:43.908941 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5daf85-9299-4dd7-8777-ab08f48644e8 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:43.972920 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac097c9-87e3-4cda-be84-a01eb5ddea7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:43.973557 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac097c9-87e3-4cda-be84-a01eb5ddea7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:43.973833 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac097c9-87e3-4cda-be84-a01eb5ddea7b admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:43.974120 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac097c9-87e3-4cda-be84-a01eb5ddea7b admin admin] PATH_INFO: `/v3/projects/39b00bcf12ec4a6fa2a4f064f245fb3c` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:43.975865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac097c9-87e3-4cda-be84-a01eb5ddea7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=39b00bcf12ec4a6fa2a4f064f245fb3c)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:43.977713 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac097c9-87e3-4cda-be84-a01eb5ddea7b admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:44.093565 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-6ac097c9-87e3-4cda-be84-a01eb5ddea7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39b00bcf12ec4a6fa2a4f064f245fb3c', 'request_id': 'req-6ac097c9-87e3-4cda-be84-a01eb5ddea7b'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:44.173113 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-6ac097c9-87e3-4cda-be84-a01eb5ddea7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39b00bcf12ec4a6fa2a4f064f245fb3c', 'request_id': 'req-6ac097c9-87e3-4cda-be84-a01eb5ddea7b'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:44.179542 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-6ac097c9-87e3-4cda-be84-a01eb5ddea7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39b00bcf12ec4a6fa2a4f064f245fb3c', 'request_id': 'req-6ac097c9-87e3-4cda-be84-a01eb5ddea7b'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:44.191860 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-6ac097c9-87e3-4cda-be84-a01eb5ddea7b admin admin] The token cache is being invalidate because project 39b00bcf12ec4a6fa2a4f064f245fb3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:59:44.192228 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-6ac097c9-87e3-4cda-be84-a01eb5ddea7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:44.193005 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-6ac097c9-87e3-4cda-be84-a01eb5ddea7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:44.195133 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1225/2469] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:59:43 2022] DELETE /identity/v3/projects/39b00bcf12ec4a6fa2a4f064f245fb3c => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:44.200246 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8070ae9-53cf-4e81-94b9-3bf6d04bc8db None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:44.200697 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8070ae9-53cf-4e81-94b9-3bf6d04bc8db None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:44.200983 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8070ae9-53cf-4e81-94b9-3bf6d04bc8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:44.222461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-d8070ae9-53cf-4e81-94b9-3bf6d04bc8db None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:44.223714 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-d8070ae9-53cf-4e81-94b9-3bf6d04bc8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:44.285539 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1245/2470] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:59:44.293542 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9616194-793f-4c2b-a73d-2472933ef2ad None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:44.293975 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9616194-793f-4c2b-a73d-2472933ef2ad None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:44.294314 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9616194-793f-4c2b-a73d-2472933ef2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:44.314513 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-a9616194-793f-4c2b-a73d-2472933ef2ad None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:44.316006 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-a9616194-793f-4c2b-a73d-2472933ef2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:44.324371 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1226/2471] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:59:44.329574 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8070ae9-53cf-4e81-94b9-3bf6d04bc8db None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:44.343963 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8070ae9-53cf-4e81-94b9-3bf6d04bc8db None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:44.348712 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b984239e-e8a9-4ed2-8f7a-031d488c4adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:44.350289 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984239e-e8a9-4ed2-8f7a-031d488c4adc admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:44.350289 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984239e-e8a9-4ed2-8f7a-031d488c4adc admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:44.350289 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984239e-e8a9-4ed2-8f7a-031d488c4adc admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:44.351150 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b984239e-e8a9-4ed2-8f7a-031d488c4adc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:59:44.351402 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b984239e-e8a9-4ed2-8f7a-031d488c4adc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:44.353031 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b984239e-e8a9-4ed2-8f7a-031d488c4adc admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:44.367252 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1246/2472] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:59:44.375651 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9616194-793f-4c2b-a73d-2472933ef2ad None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:44.384877 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9616194-793f-4c2b-a73d-2472933ef2ad None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:44.389732 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311bdde-f7ff-483b-8600-e0b6442c1c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:44.390527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311bdde-f7ff-483b-8600-e0b6442c1c13 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:44.390654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311bdde-f7ff-483b-8600-e0b6442c1c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:44.390861 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d311bdde-f7ff-483b-8600-e0b6442c1c13 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:44.391503 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d311bdde-f7ff-483b-8600-e0b6442c1c13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:44.393440 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d311bdde-f7ff-483b-8600-e0b6442c1c13 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:44.493588 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1227/2473] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:59:44 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:59:44.498005 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b984239e-e8a9-4ed2-8f7a-031d488c4adc admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:44.508218 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b984239e-e8a9-4ed2-8f7a-031d488c4adc admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:44.566516 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11740cbc-7e28-4b47-bc54-33f0b0a79bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:44.567140 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11740cbc-7e28-4b47-bc54-33f0b0a79bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:44.567413 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11740cbc-7e28-4b47-bc54-33f0b0a79bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:44.567684 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11740cbc-7e28-4b47-bc54-33f0b0a79bd0 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:44.568338 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11740cbc-7e28-4b47-bc54-33f0b0a79bd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:44.570231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11740cbc-7e28-4b47-bc54-33f0b0a79bd0 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:44.692370 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1247/2474] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:59:44 2022] POST /identity/v3/users => generated 287 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:59:44.698138 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311bdde-f7ff-483b-8600-e0b6442c1c13 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:44.712224 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d311bdde-f7ff-483b-8600-e0b6442c1c13 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:44.717475 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff9657d-5f09-4434-a8d6-b2cd8b7dadf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:44.718333 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9657d-5f09-4434-a8d6-b2cd8b7dadf6 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:44.718540 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9657d-5f09-4434-a8d6-b2cd8b7dadf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:44.718945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9657d-5f09-4434-a8d6-b2cd8b7dadf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:44.719602 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff9657d-5f09-4434-a8d6-b2cd8b7dadf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:44.721346 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff9657d-5f09-4434-a8d6-b2cd8b7dadf6 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:44.732709 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1228/2475] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:59:44.736768 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11740cbc-7e28-4b47-bc54-33f0b0a79bd0 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:44.747018 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11740cbc-7e28-4b47-bc54-33f0b0a79bd0 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:44.751728 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff4862-0e82-425c-921a-57caffdec693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:44.752410 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff4862-0e82-425c-921a-57caffdec693 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:44.758012 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff4862-0e82-425c-921a-57caffdec693 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:44.758012 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff4862-0e82-425c-921a-57caffdec693 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:44.758012 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff4862-0e82-425c-921a-57caffdec693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:44.758012 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff4862-0e82-425c-921a-57caffdec693 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:44.765754 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1248/2476] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 03:59:44.769930 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff9657d-5f09-4434-a8d6-b2cd8b7dadf6 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:44.781276 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff9657d-5f09-4434-a8d6-b2cd8b7dadf6 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:44.786404 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159862c-982e-4788-8fd4-761b325157d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:44.787068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159862c-982e-4788-8fd4-761b325157d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:44.787354 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159862c-982e-4788-8fd4-761b325157d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:44.787631 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159862c-982e-4788-8fd4-761b325157d2 admin admin] PATH_INFO: `/v3/projects/7715f95edb1449dba75e35ffb3e5ad9b/users/719735a47aee4ce185a54b5d8e6b18f4/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:44.815555 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159862c-982e-4788-8fd4-761b325157d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7715f95edb1449dba75e35ffb3e5ad9b, user_id=719735a47aee4ce185a54b5d8e6b18f4, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:44.815555 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159862c-982e-4788-8fd4-761b325157d2 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:44.922640 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:44.928474 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1229/2477] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:59:44 2022] PUT /identity/v3/projects/7715f95edb1449dba75e35ffb3e5ad9b/users/719735a47aee4ce185a54b5d8e6b18f4/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:44.940098 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baafce7-3b9e-4af5-a013-7ac53e37db68 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:44.940384 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baafce7-3b9e-4af5-a013-7ac53e37db68 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:44.940656 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baafce7-3b9e-4af5-a013-7ac53e37db68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:44.972843 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-6baafce7-3b9e-4af5-a013-7ac53e37db68 None None] MFA Rules not processed for user `719735a47aee4ce185a54b5d8e6b18f4`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:44.974057 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6baafce7-3b9e-4af5-a013-7ac53e37db68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:44.978201 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:45.038459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1249/2478] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:59:45.049878 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159862c-982e-4788-8fd4-761b325157d2 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:45.056047 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-8159862c-982e-4788-8fd4-761b325157d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:45.108233 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159862c-982e-4788-8fd4-761b325157d2 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:45.114628 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a52f2-4e52-4bf5-9e6c-61a97295ac23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:45.114628 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a52f2-4e52-4bf5-9e6c-61a97295ac23 service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:45.114628 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a52f2-4e52-4bf5-9e6c-61a97295ac23 service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:45.114628 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a52f2-4e52-4bf5-9e6c-61a97295ac23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:45.121407 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a52f2-4e52-4bf5-9e6c-61a97295ac23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:45.122837 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a52f2-4e52-4bf5-9e6c-61a97295ac23 service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:45.133298 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1230/2479] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:59:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 03:59:45.156328 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baafce7-3b9e-4af5-a013-7ac53e37db68 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:45.224906 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baafce7-3b9e-4af5-a013-7ac53e37db68 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:45.229464 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4d72b-5ac7-4632-b8a7-6e71c797e597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:45.230089 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4d72b-5ac7-4632-b8a7-6e71c797e597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:45.230367 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4d72b-5ac7-4632-b8a7-6e71c797e597 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:45.230639 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4d72b-5ac7-4632-b8a7-6e71c797e597 admin admin] PATH_INFO: `/v3/users/719735a47aee4ce185a54b5d8e6b18f4` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:45.231884 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4d72b-5ac7-4632-b8a7-6e71c797e597 admin admin] RBAC: Authorizing `identity:delete_user(user_id=719735a47aee4ce185a54b5d8e6b18f4)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:45.233811 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4d72b-5ac7-4632-b8a7-6e71c797e597 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:45.414472 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-c2c4d72b-5ac7-4632-b8a7-6e71c797e597 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '719735a47aee4ce185a54b5d8e6b18f4'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:45.415142 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-c2c4d72b-5ac7-4632-b8a7-6e71c797e597 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '719735a47aee4ce185a54b5d8e6b18f4'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:45.480089 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-c2c4d72b-5ac7-4632-b8a7-6e71c797e597 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '719735a47aee4ce185a54b5d8e6b18f4'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:45.493416 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-c2c4d72b-5ac7-4632-b8a7-6e71c797e597 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '719735a47aee4ce185a54b5d8e6b18f4'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:45.494070 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-c2c4d72b-5ac7-4632-b8a7-6e71c797e597 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '719735a47aee4ce185a54b5d8e6b18f4'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:45.506403 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1250/2480] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:59:45 2022] DELETE /identity/v3/users/719735a47aee4ce185a54b5d8e6b18f4 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:45.511469 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286825e-e456-4980-aee4-e7298e84af64 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:45.512055 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286825e-e456-4980-aee4-e7298e84af64 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:45.512055 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286825e-e456-4980-aee4-e7298e84af64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:45.531238 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-d286825e-e456-4980-aee4-e7298e84af64 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:45.532473 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-d286825e-e456-4980-aee4-e7298e84af64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:45.589531 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1231/2481] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:59:45.599129 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4d72b-5ac7-4632-b8a7-6e71c797e597 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:45.600622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-c2c4d72b-5ac7-4632-b8a7-6e71c797e597 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:45.628808 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:45.657445 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4d72b-5ac7-4632-b8a7-6e71c797e597 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:45.661130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09f4a73-edd8-4557-8895-b7141f1e473d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:45.661733 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f4a73-edd8-4557-8895-b7141f1e473d service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:45.662019 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f4a73-edd8-4557-8895-b7141f1e473d service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:45.662296 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09f4a73-edd8-4557-8895-b7141f1e473d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:45.673049 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f4a73-edd8-4557-8895-b7141f1e473d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:45.675080 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09f4a73-edd8-4557-8895-b7141f1e473d service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:45.687234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1251/2482] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:59:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:59:46.135685 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d286825e-e456-4980-aee4-e7298e84af64 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:46.137214 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-d286825e-e456-4980-aee4-e7298e84af64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:46.149124 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d286825e-e456-4980-aee4-e7298e84af64 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:46.160107 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b9ab3-551e-4c7a-bd04-1d977f2275d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:46.161284 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b9ab3-551e-4c7a-bd04-1d977f2275d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:46.161628 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b9ab3-551e-4c7a-bd04-1d977f2275d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:46.161967 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b9ab3-551e-4c7a-bd04-1d977f2275d5 admin admin] PATH_INFO: `/v3/projects/7715f95edb1449dba75e35ffb3e5ad9b` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:46.163167 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98b9ab3-551e-4c7a-bd04-1d977f2275d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7715f95edb1449dba75e35ffb3e5ad9b)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:46.165211 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98b9ab3-551e-4c7a-bd04-1d977f2275d5 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:46.276932 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-d98b9ab3-551e-4c7a-bd04-1d977f2275d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7715f95edb1449dba75e35ffb3e5ad9b', 'request_id': 'req-d98b9ab3-551e-4c7a-bd04-1d977f2275d5'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:46.336385 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-d98b9ab3-551e-4c7a-bd04-1d977f2275d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7715f95edb1449dba75e35ffb3e5ad9b', 'request_id': 'req-d98b9ab3-551e-4c7a-bd04-1d977f2275d5'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:46.342769 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-d98b9ab3-551e-4c7a-bd04-1d977f2275d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7715f95edb1449dba75e35ffb3e5ad9b', 'request_id': 'req-d98b9ab3-551e-4c7a-bd04-1d977f2275d5'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:46.356753 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-d98b9ab3-551e-4c7a-bd04-1d977f2275d5 admin admin] The token cache is being invalidate because project 7715f95edb1449dba75e35ffb3e5ad9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:59:46.357047 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-d98b9ab3-551e-4c7a-bd04-1d977f2275d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:46.357745 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-d98b9ab3-551e-4c7a-bd04-1d977f2275d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:46.359406 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1232/2483] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:59:46 2022] DELETE /identity/v3/projects/7715f95edb1449dba75e35ffb3e5ad9b => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 17 03:59:46.364606 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cd98b-2e6c-4e01-abc7-e44a4ce1e5a5 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:46.365090 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cd98b-2e6c-4e01-abc7-e44a4ce1e5a5 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:46.365192 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624cd98b-2e6c-4e01-abc7-e44a4ce1e5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:46.386186 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-624cd98b-2e6c-4e01-abc7-e44a4ce1e5a5 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:46.387380 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-624cd98b-2e6c-4e01-abc7-e44a4ce1e5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:46.448961 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1252/2484] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 03:59:46.456898 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300b257-f956-4fba-9941-1a1c4797b1be None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:46.457184 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300b257-f956-4fba-9941-1a1c4797b1be None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:46.457460 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300b257-f956-4fba-9941-1a1c4797b1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:46.477921 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-1300b257-f956-4fba-9941-1a1c4797b1be None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:46.479141 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-1300b257-f956-4fba-9941-1a1c4797b1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:46.487360 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1233/2485] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:59:46.492406 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624cd98b-2e6c-4e01-abc7-e44a4ce1e5a5 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:46.502196 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624cd98b-2e6c-4e01-abc7-e44a4ce1e5a5 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:46.506869 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db55d0-485e-47a9-8097-b03e269fefdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:46.507471 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db55d0-485e-47a9-8097-b03e269fefdc admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:46.507784 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db55d0-485e-47a9-8097-b03e269fefdc admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:46.508062 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db55d0-485e-47a9-8097-b03e269fefdc admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:46.508764 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db55d0-485e-47a9-8097-b03e269fefdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:59:46.509073 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db55d0-485e-47a9-8097-b03e269fefdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:46.510657 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db55d0-485e-47a9-8097-b03e269fefdc admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:46.522211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1253/2486] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:59:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:59:46.526410 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1300b257-f956-4fba-9941-1a1c4797b1be None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:46.535873 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1300b257-f956-4fba-9941-1a1c4797b1be None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:46.541117 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b028477-96ef-4423-988f-d758acb71844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:46.543562 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b028477-96ef-4423-988f-d758acb71844 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:46.543562 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b028477-96ef-4423-988f-d758acb71844 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:46.543562 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b028477-96ef-4423-988f-d758acb71844 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:46.543562 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b028477-96ef-4423-988f-d758acb71844 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:46.544756 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b028477-96ef-4423-988f-d758acb71844 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:46.562261 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:46.684167 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1234/2487] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:59:46 2022] POST /identity/v3/projects => generated 383 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:59:46.688211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db55d0-485e-47a9-8097-b03e269fefdc admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:46.698422 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db55d0-485e-47a9-8097-b03e269fefdc admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:46.758472 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690124f3-e9cb-4640-9c6e-4d8f9f5eb85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:46.759086 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690124f3-e9cb-4640-9c6e-4d8f9f5eb85c admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:46.759367 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690124f3-e9cb-4640-9c6e-4d8f9f5eb85c admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:46.759642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690124f3-e9cb-4640-9c6e-4d8f9f5eb85c admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:46.760303 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690124f3-e9cb-4640-9c6e-4d8f9f5eb85c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:46.762215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690124f3-e9cb-4640-9c6e-4d8f9f5eb85c admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:46.882205 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1254/2488] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:59:46 2022] POST /identity/v3/users => generated 292 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:59:46.886310 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b028477-96ef-4423-988f-d758acb71844 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:46.896701 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b028477-96ef-4423-988f-d758acb71844 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:46.903125 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22824698-6866-4b4a-90d7-8079983443b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:46.903726 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22824698-6866-4b4a-90d7-8079983443b9 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:46.904030 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22824698-6866-4b4a-90d7-8079983443b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:46.904301 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22824698-6866-4b4a-90d7-8079983443b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:46.904939 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22824698-6866-4b4a-90d7-8079983443b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:46.906495 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22824698-6866-4b4a-90d7-8079983443b9 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:46.919974 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1235/2489] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:59:46.921231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690124f3-e9cb-4640-9c6e-4d8f9f5eb85c admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:46.931973 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690124f3-e9cb-4640-9c6e-4d8f9f5eb85c admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:46.936206 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d113f-e382-4a7d-8319-83289b02dff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:46.936804 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d113f-e382-4a7d-8319-83289b02dff6 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:46.937086 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d113f-e382-4a7d-8319-83289b02dff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:46.937360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d113f-e382-4a7d-8319-83289b02dff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:46.937996 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d113f-e382-4a7d-8319-83289b02dff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:46.939604 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d113f-e382-4a7d-8319-83289b02dff6 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:46.952611 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1255/2490] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:59:46.956100 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22824698-6866-4b4a-90d7-8079983443b9 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:46.966677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22824698-6866-4b4a-90d7-8079983443b9 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:46.971144 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e79281-6e35-4515-955a-fee3257d8161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:46.971791 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e79281-6e35-4515-955a-fee3257d8161 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:46.972090 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e79281-6e35-4515-955a-fee3257d8161 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:46.972341 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e79281-6e35-4515-955a-fee3257d8161 admin admin] PATH_INFO: `/v3/projects/f1fea935de384bfa81953ec336e06b7f/users/b86807317d1247c4a9afdf4c1afa3a2d/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:46.987112 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e79281-6e35-4515-955a-fee3257d8161 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1fea935de384bfa81953ec336e06b7f, user_id=b86807317d1247c4a9afdf4c1afa3a2d, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:46.989758 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e79281-6e35-4515-955a-fee3257d8161 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:47.122358 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1236/2491] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:59:46 2022] PUT /identity/v3/projects/f1fea935de384bfa81953ec336e06b7f/users/b86807317d1247c4a9afdf4c1afa3a2d/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:47.127378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc4a87-b2dc-42d5-a499-ad16d264574a None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:47.127691 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc4a87-b2dc-42d5-a499-ad16d264574a None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:47.127994 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc4a87-b2dc-42d5-a499-ad16d264574a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:47.148130 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-91cc4a87-b2dc-42d5-a499-ad16d264574a None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:47.150632 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-91cc4a87-b2dc-42d5-a499-ad16d264574a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:47.181102 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:47.212105 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1256/2492] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:59:47.222567 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e79281-6e35-4515-955a-fee3257d8161 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:47.224686 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-45e79281-6e35-4515-955a-fee3257d8161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:47.282637 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e79281-6e35-4515-955a-fee3257d8161 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:47.286320 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da2d28-481f-4133-83fb-e9ca64f3ddcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:47.286917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da2d28-481f-4133-83fb-e9ca64f3ddcf service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:47.287200 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da2d28-481f-4133-83fb-e9ca64f3ddcf service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:47.287470 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da2d28-481f-4133-83fb-e9ca64f3ddcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:47.297666 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da2d28-481f-4133-83fb-e9ca64f3ddcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:47.299875 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da2d28-481f-4133-83fb-e9ca64f3ddcf service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:47.310633 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1237/2493] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:59:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:59:49.783155 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cc4a87-b2dc-42d5-a499-ad16d264574a None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:49.784892 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-91cc4a87-b2dc-42d5-a499-ad16d264574a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:49.798786 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cc4a87-b2dc-42d5-a499-ad16d264574a None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:49.804220 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126ad99-f80b-414e-a814-f4487b411eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:49.805041 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126ad99-f80b-414e-a814-f4487b411eff admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:49.805386 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126ad99-f80b-414e-a814-f4487b411eff admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:49.805729 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e126ad99-f80b-414e-a814-f4487b411eff admin admin] PATH_INFO: `/v3/users/f0717a2295cb463e8f0645830551402d` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:49.807175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e126ad99-f80b-414e-a814-f4487b411eff admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0717a2295cb463e8f0645830551402d)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:49.809265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e126ad99-f80b-414e-a814-f4487b411eff admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:50.021622 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-e126ad99-f80b-414e-a814-f4487b411eff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0717a2295cb463e8f0645830551402d'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:50.022971 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-e126ad99-f80b-414e-a814-f4487b411eff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0717a2295cb463e8f0645830551402d'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:50.096661 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-e126ad99-f80b-414e-a814-f4487b411eff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0717a2295cb463e8f0645830551402d'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:50.126963 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-e126ad99-f80b-414e-a814-f4487b411eff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0717a2295cb463e8f0645830551402d'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:50.128189 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-e126ad99-f80b-414e-a814-f4487b411eff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0717a2295cb463e8f0645830551402d'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:50.144314 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1257/2494] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 03:59:49 2022] DELETE /identity/v3/users/f0717a2295cb463e8f0645830551402d => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:50.146340 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa04fe-c077-4b2a-bd8c-c9d7d9b6eb72 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:50.146826 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa04fe-c077-4b2a-bd8c-c9d7d9b6eb72 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:50.146826 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa04fe-c077-4b2a-bd8c-c9d7d9b6eb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:50.172382 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-c7aa04fe-c077-4b2a-bd8c-c9d7d9b6eb72 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:50.173959 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c7aa04fe-c077-4b2a-bd8c-c9d7d9b6eb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:50.246556 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1238/2495] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:59:50.253007 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126ad99-f80b-414e-a814-f4487b411eff admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:50.255033 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-e126ad99-f80b-414e-a814-f4487b411eff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:50.342370 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e126ad99-f80b-414e-a814-f4487b411eff admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:50.347718 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6040a072-8ae3-43fa-a12b-8f67c7b5cf9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:50.348452 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6040a072-8ae3-43fa-a12b-8f67c7b5cf9c service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:50.348733 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6040a072-8ae3-43fa-a12b-8f67c7b5cf9c service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:50.349000 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6040a072-8ae3-43fa-a12b-8f67c7b5cf9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:50.359620 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6040a072-8ae3-43fa-a12b-8f67c7b5cf9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:50.362020 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6040a072-8ae3-43fa-a12b-8f67c7b5cf9c service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:50.371928 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1258/2496] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:59:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:59:50.875327 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa04fe-c077-4b2a-bd8c-c9d7d9b6eb72 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:50.876874 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c7aa04fe-c077-4b2a-bd8c-c9d7d9b6eb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:50.890905 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa04fe-c077-4b2a-bd8c-c9d7d9b6eb72 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:50.911871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a71c56-9266-432d-b5de-b24840c2628b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:50.911871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a71c56-9266-432d-b5de-b24840c2628b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:50.911871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a71c56-9266-432d-b5de-b24840c2628b admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:50.911871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a71c56-9266-432d-b5de-b24840c2628b admin admin] PATH_INFO: `/v3/projects/2a44d7e011fe4a53a863bfe9b513ab27` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:50.914631 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a71c56-9266-432d-b5de-b24840c2628b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a44d7e011fe4a53a863bfe9b513ab27)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:50.917324 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a71c56-9266-432d-b5de-b24840c2628b admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:51.037179 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f6a71c56-9266-432d-b5de-b24840c2628b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a44d7e011fe4a53a863bfe9b513ab27', 'request_id': 'req-f6a71c56-9266-432d-b5de-b24840c2628b'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:51.131596 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f6a71c56-9266-432d-b5de-b24840c2628b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a44d7e011fe4a53a863bfe9b513ab27', 'request_id': 'req-f6a71c56-9266-432d-b5de-b24840c2628b'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:51.135001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:51.139361 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f6a71c56-9266-432d-b5de-b24840c2628b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a44d7e011fe4a53a863bfe9b513ab27', 'request_id': 'req-f6a71c56-9266-432d-b5de-b24840c2628b'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:51.160032 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f6a71c56-9266-432d-b5de-b24840c2628b admin admin] The token cache is being invalidate because project 2a44d7e011fe4a53a863bfe9b513ab27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 03:59:51.160725 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f6a71c56-9266-432d-b5de-b24840c2628b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:51.161587 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-f6a71c56-9266-432d-b5de-b24840c2628b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 03:59:51.163771 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1239/2497] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 03:59:50 2022] DELETE /identity/v3/projects/2a44d7e011fe4a53a863bfe9b513ab27 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:51.168905 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59165285-1815-47d2-b4e1-c8de66b36cab None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:51.169998 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59165285-1815-47d2-b4e1-c8de66b36cab None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:51.169998 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59165285-1815-47d2-b4e1-c8de66b36cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:51.195577 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-59165285-1815-47d2-b4e1-c8de66b36cab None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:51.196976 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-59165285-1815-47d2-b4e1-c8de66b36cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:51.248133 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:51.266831 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1259/2498] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:59:51.276552 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748fa33-17eb-43ee-8a89-91aae7d692ee None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:51.276902 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748fa33-17eb-43ee-8a89-91aae7d692ee None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:51.277281 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748fa33-17eb-43ee-8a89-91aae7d692ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:51.301332 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-9748fa33-17eb-43ee-8a89-91aae7d692ee None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:51.302619 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-9748fa33-17eb-43ee-8a89-91aae7d692ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:51.314063 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1240/2499] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:59:51.321778 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59165285-1815-47d2-b4e1-c8de66b36cab None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:51.333629 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59165285-1815-47d2-b4e1-c8de66b36cab None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:51.341992 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39adc15f-11d6-4b2f-a6fb-54b21ee1b281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:51.342732 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39adc15f-11d6-4b2f-a6fb-54b21ee1b281 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:51.343106 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39adc15f-11d6-4b2f-a6fb-54b21ee1b281 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:51.343494 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39adc15f-11d6-4b2f-a6fb-54b21ee1b281 admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:51.344807 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39adc15f-11d6-4b2f-a6fb-54b21ee1b281 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 03:59:51.350127 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39adc15f-11d6-4b2f-a6fb-54b21ee1b281 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:51.353077 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39adc15f-11d6-4b2f-a6fb-54b21ee1b281 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:51.371642 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1260/2500] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 03:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 03:59:51.372994 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748fa33-17eb-43ee-8a89-91aae7d692ee None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:51.383266 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748fa33-17eb-43ee-8a89-91aae7d692ee None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:51.388228 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649429c5-da1f-4efa-b737-1c3645cc2d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:51.388838 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649429c5-da1f-4efa-b737-1c3645cc2d3d admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:51.389122 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649429c5-da1f-4efa-b737-1c3645cc2d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:51.389401 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649429c5-da1f-4efa-b737-1c3645cc2d3d admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:51.390044 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649429c5-da1f-4efa-b737-1c3645cc2d3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:51.391985 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649429c5-da1f-4efa-b737-1c3645cc2d3d admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:51.487712 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1241/2501] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:59:51 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:59:51.491766 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39adc15f-11d6-4b2f-a6fb-54b21ee1b281 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:51.505937 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39adc15f-11d6-4b2f-a6fb-54b21ee1b281 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:51.581646 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39771205-8773-429b-8a2f-01f26c0db13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:51.582269 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39771205-8773-429b-8a2f-01f26c0db13d admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:51.582551 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39771205-8773-429b-8a2f-01f26c0db13d admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:51.582826 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39771205-8773-429b-8a2f-01f26c0db13d admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:51.583467 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39771205-8773-429b-8a2f-01f26c0db13d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:51.585468 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39771205-8773-429b-8a2f-01f26c0db13d admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:51.692440 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1261/2502] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:59:51 2022] POST /identity/v3/users => generated 286 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:59:51.696644 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649429c5-da1f-4efa-b737-1c3645cc2d3d admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:51.714506 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649429c5-da1f-4efa-b737-1c3645cc2d3d admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:51.720555 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bd382-0af3-4501-a451-2eb00d0cb999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:51.733802 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bd382-0af3-4501-a451-2eb00d0cb999 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:51.733802 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bd382-0af3-4501-a451-2eb00d0cb999 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:51.733802 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79bd382-0af3-4501-a451-2eb00d0cb999 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:51.733802 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bd382-0af3-4501-a451-2eb00d0cb999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:51.733802 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79bd382-0af3-4501-a451-2eb00d0cb999 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:51.742162 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1242/2503] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:59:51.743244 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39771205-8773-429b-8a2f-01f26c0db13d admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:51.765386 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39771205-8773-429b-8a2f-01f26c0db13d admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:51.771307 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4212c1c-5677-4ef0-b258-f9288bec2f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:51.776084 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4212c1c-5677-4ef0-b258-f9288bec2f17 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:51.776084 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4212c1c-5677-4ef0-b258-f9288bec2f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:51.776084 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4212c1c-5677-4ef0-b258-f9288bec2f17 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:51.776084 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4212c1c-5677-4ef0-b258-f9288bec2f17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:51.776084 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4212c1c-5677-4ef0-b258-f9288bec2f17 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:51.789501 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1262/2504] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:59:51.794225 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bd382-0af3-4501-a451-2eb00d0cb999 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:51.805917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79bd382-0af3-4501-a451-2eb00d0cb999 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:51.811035 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a14781-adad-41f1-9f2e-caa7eec546ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:51.811949 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a14781-adad-41f1-9f2e-caa7eec546ff admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:51.812335 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a14781-adad-41f1-9f2e-caa7eec546ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:51.812721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a14781-adad-41f1-9f2e-caa7eec546ff admin admin] PATH_INFO: `/v3/projects/a64255e69ce9468e9a7bef2297b17725/users/926a7f570adb46debe57caefead0e17b/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:51.831442 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a14781-adad-41f1-9f2e-caa7eec546ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=a64255e69ce9468e9a7bef2297b17725, user_id=926a7f570adb46debe57caefead0e17b, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:51.834422 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a14781-adad-41f1-9f2e-caa7eec546ff admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:51.946006 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1243/2505] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:59:51 2022] PUT /identity/v3/projects/a64255e69ce9468e9a7bef2297b17725/users/926a7f570adb46debe57caefead0e17b/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:51.951934 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8421b-643a-4218-8722-92d1e8bcffa7 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:51.952343 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8421b-643a-4218-8722-92d1e8bcffa7 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:51.952744 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8421b-643a-4218-8722-92d1e8bcffa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:51.976747 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-6df8421b-643a-4218-8722-92d1e8bcffa7 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:51.978446 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6df8421b-643a-4218-8722-92d1e8bcffa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:52.048066 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1263/2506] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:59:52.058184 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a14781-adad-41f1-9f2e-caa7eec546ff admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:52.060068 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-09a14781-adad-41f1-9f2e-caa7eec546ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:52.137098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a14781-adad-41f1-9f2e-caa7eec546ff admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:52.141274 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d9d71-7ba0-4ff6-bbb0-f9419aeea46b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:52.146201 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d9d71-7ba0-4ff6-bbb0-f9419aeea46b service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:52.146201 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d9d71-7ba0-4ff6-bbb0-f9419aeea46b service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:52.146201 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d9d71-7ba0-4ff6-bbb0-f9419aeea46b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:52.154594 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214d9d71-7ba0-4ff6-bbb0-f9419aeea46b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:52.156096 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214d9d71-7ba0-4ff6-bbb0-f9419aeea46b service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:52.166686 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1244/2507] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 03:59:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 03:59:52.709331 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de69bb-089b-4497-8a40-2421e2310953 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:52.710430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de69bb-089b-4497-8a40-2421e2310953 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:52.710430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66de69bb-089b-4497-8a40-2421e2310953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:52.744278 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-66de69bb-089b-4497-8a40-2421e2310953 None None] MFA Rules not processed for user `b86807317d1247c4a9afdf4c1afa3a2d`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:52.745557 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-66de69bb-089b-4497-8a40-2421e2310953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:52.799141 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:52.828623 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d9d71-7ba0-4ff6-bbb0-f9419aeea46b service neutron] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:52.830329 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-214d9d71-7ba0-4ff6-bbb0-f9419aeea46b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:52.831869 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1264/2508] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:59:52.845430 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d9d71-7ba0-4ff6-bbb0-f9419aeea46b service neutron] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:52.853157 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d58a3-6c04-4388-97c4-ae9b304aa1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:52.859488 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d58a3-6c04-4388-97c4-ae9b304aa1d0 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:52.859488 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d58a3-6c04-4388-97c4-ae9b304aa1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:52.859488 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0d58a3-6c04-4388-97c4-ae9b304aa1d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:52.859488 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d58a3-6c04-4388-97c4-ae9b304aa1d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:52.859488 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0d58a3-6c04-4388-97c4-ae9b304aa1d0 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:52.980450 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1245/2509] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:59:52 2022] POST /identity/v3/projects => generated 383 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:59:52.984672 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de69bb-089b-4497-8a40-2421e2310953 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:52.996292 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66de69bb-089b-4497-8a40-2421e2310953 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:53.058152 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9849f-87d0-4e42-8c5f-42ea3e722636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:53.058780 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9849f-87d0-4e42-8c5f-42ea3e722636 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:53.059065 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9849f-87d0-4e42-8c5f-42ea3e722636 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:53.059335 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9849f-87d0-4e42-8c5f-42ea3e722636 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:53.060022 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9849f-87d0-4e42-8c5f-42ea3e722636 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:53.061940 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb9849f-87d0-4e42-8c5f-42ea3e722636 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:53.205775 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1265/2510] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:59:52 2022] POST /identity/v3/users => generated 298 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 17 03:59:53.210411 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d58a3-6c04-4388-97c4-ae9b304aa1d0 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:53.220836 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0d58a3-6c04-4388-97c4-ae9b304aa1d0 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:53.225358 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c2bbe-1c4a-4dec-a026-c26c5a08d9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:53.225954 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c2bbe-1c4a-4dec-a026-c26c5a08d9a6 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:53.226237 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c2bbe-1c4a-4dec-a026-c26c5a08d9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:53.226511 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9c2bbe-1c4a-4dec-a026-c26c5a08d9a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:53.227168 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9c2bbe-1c4a-4dec-a026-c26c5a08d9a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:53.228763 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9c2bbe-1c4a-4dec-a026-c26c5a08d9a6 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:53.240754 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1246/2511] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:59:53.244902 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9849f-87d0-4e42-8c5f-42ea3e722636 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:53.255260 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb9849f-87d0-4e42-8c5f-42ea3e722636 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:53.261337 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842977a-a985-44f8-9663-080be288e197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:53.262008 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842977a-a985-44f8-9663-080be288e197 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:53.262288 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842977a-a985-44f8-9663-080be288e197 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:53.262562 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5842977a-a985-44f8-9663-080be288e197 admin admin] PATH_INFO: `/v3/projects/ccc8c4e5d2864b9cb530c524185c792e/users/1f7a7628896e4005b6e97b02b3266a89/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:53.282177 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5842977a-a985-44f8-9663-080be288e197 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccc8c4e5d2864b9cb530c524185c792e, user_id=1f7a7628896e4005b6e97b02b3266a89, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:53.286102 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5842977a-a985-44f8-9663-080be288e197 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:53.356745 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1266/2512] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:59:53 2022] PUT /identity/v3/projects/ccc8c4e5d2864b9cb530c524185c792e/users/1f7a7628896e4005b6e97b02b3266a89/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:53.360762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c2bbe-1c4a-4dec-a026-c26c5a08d9a6 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:53.371113 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9c2bbe-1c4a-4dec-a026-c26c5a08d9a6 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:53.427546 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 03:59:53.436288 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c720bf-568b-4bac-8b35-66b936fbed10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:53.436811 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c720bf-568b-4bac-8b35-66b936fbed10 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:53.437094 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c720bf-568b-4bac-8b35-66b936fbed10 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:53.437366 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c720bf-568b-4bac-8b35-66b936fbed10 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:53.438010 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c720bf-568b-4bac-8b35-66b936fbed10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:53.439594 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c720bf-568b-4bac-8b35-66b936fbed10 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:53.449850 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1247/2513] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:59:53 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:59:53.455307 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842977a-a985-44f8-9663-080be288e197 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:53.465336 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5842977a-a985-44f8-9663-080be288e197 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:53.470395 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f1ba0-8d21-4c3c-8ef3-ac717ece4c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:53.471093 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f1ba0-8d21-4c3c-8ef3-ac717ece4c6a admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:53.471376 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f1ba0-8d21-4c3c-8ef3-ac717ece4c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:53.471607 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81f1ba0-8d21-4c3c-8ef3-ac717ece4c6a admin admin] PATH_INFO: `/v3/projects/ccc8c4e5d2864b9cb530c524185c792e/users/1f7a7628896e4005b6e97b02b3266a89/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:53.474174 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f1ba0-8d21-4c3c-8ef3-ac717ece4c6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccc8c4e5d2864b9cb530c524185c792e, user_id=1f7a7628896e4005b6e97b02b3266a89, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:53.476695 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81f1ba0-8d21-4c3c-8ef3-ac717ece4c6a admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:53.492367 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1267/2514] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:59:53 2022] PUT /identity/v3/projects/ccc8c4e5d2864b9cb530c524185c792e/users/1f7a7628896e4005b6e97b02b3266a89/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 03:59:59.260708 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba45f0-3f61-48e0-a30d-7e4d6ac5166d None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:59.262367 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba45f0-3f61-48e0-a30d-7e4d6ac5166d None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:59.263497 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba45f0-3f61-48e0-a30d-7e4d6ac5166d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:59.304234 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-1bba45f0-3f61-48e0-a30d-7e4d6ac5166d None None] MFA Rules not processed for user `926a7f570adb46debe57caefead0e17b`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 03:59:59.305781 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-1bba45f0-3f61-48e0-a30d-7e4d6ac5166d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 03:59:59.410696 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1248/2515] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 03:59:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 03:59:59.415339 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f1ba0-8d21-4c3c-8ef3-ac717ece4c6a admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:59.526032 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81f1ba0-8d21-4c3c-8ef3-ac717ece4c6a admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:59.535259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac05c2b-40d4-4c70-88e8-d867ea937445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:59.535259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac05c2b-40d4-4c70-88e8-d867ea937445 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:59.535259 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac05c2b-40d4-4c70-88e8-d867ea937445 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:59.535836 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac05c2b-40d4-4c70-88e8-d867ea937445 admin admin] PATH_INFO: `/v3/projects` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:59.536523 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac05c2b-40d4-4c70-88e8-d867ea937445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:59.538688 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac05c2b-40d4-4c70-88e8-d867ea937445 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:59.692741 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1268/2516] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 03:59:59 2022] POST /identity/v3/projects => generated 373 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:59:59.698655 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba45f0-3f61-48e0-a30d-7e4d6ac5166d None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:59.735486 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba45f0-3f61-48e0-a30d-7e4d6ac5166d None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:59.812561 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023e184-a27d-4c32-af7c-e8c7d00eccc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:59.813321 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023e184-a27d-4c32-af7c-e8c7d00eccc7 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:59.813731 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023e184-a27d-4c32-af7c-e8c7d00eccc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:59.814111 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023e184-a27d-4c32-af7c-e8c7d00eccc7 admin admin] PATH_INFO: `/v3/users` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:59.814850 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1023e184-a27d-4c32-af7c-e8c7d00eccc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:59.816920 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1023e184-a27d-4c32-af7c-e8c7d00eccc7 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:59.894172 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1249/2517] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 03:59:59 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 03:59:59.898025 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac05c2b-40d4-4c70-88e8-d867ea937445 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:59.911049 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac05c2b-40d4-4c70-88e8-d867ea937445 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:59.916746 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f13125-a80f-48b6-9cc8-84b891ab97de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:59.917702 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f13125-a80f-48b6-9cc8-84b891ab97de admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:59.918204 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f13125-a80f-48b6-9cc8-84b891ab97de admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:59.918575 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f13125-a80f-48b6-9cc8-84b891ab97de admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:59.919564 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f13125-a80f-48b6-9cc8-84b891ab97de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:59.921558 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f13125-a80f-48b6-9cc8-84b891ab97de admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 03:59:59.938435 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1269/2518] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 03:59:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 03:59:59.939322 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023e184-a27d-4c32-af7c-e8c7d00eccc7 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 03:59:59.951269 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023e184-a27d-4c32-af7c-e8c7d00eccc7 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 03:59:59.956340 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59683568-e465-4bee-a8e3-cdc80b964b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 03:59:59.957201 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59683568-e465-4bee-a8e3-cdc80b964b9d admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 03:59:59.957744 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59683568-e465-4bee-a8e3-cdc80b964b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 03:59:59.958124 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59683568-e465-4bee-a8e3-cdc80b964b9d admin admin] PATH_INFO: `/v3/projects/2f05de57855e405cb4ff423353bc85c7/users/de56df051ac546dcb3f5c1d43e348f7e/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 03:59:59.981166 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59683568-e465-4bee-a8e3-cdc80b964b9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f05de57855e405cb4ff423353bc85c7, user_id=de56df051ac546dcb3f5c1d43e348f7e, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 03:59:59.983153 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59683568-e465-4bee-a8e3-cdc80b964b9d admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:00:00.068097 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1250/2519] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 03:59:59 2022] PUT /identity/v3/projects/2f05de57855e405cb4ff423353bc85c7/users/de56df051ac546dcb3f5c1d43e348f7e/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 04:00:00.072902 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f13125-a80f-48b6-9cc8-84b891ab97de admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:00:00.108890 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f13125-a80f-48b6-9cc8-84b891ab97de admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:00:00.125317 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:00:00.212492 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf54381e-1bd4-4961-8530-4c975ddcf643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:00:00.212775 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54381e-1bd4-4961-8530-4c975ddcf643 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:00.213067 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54381e-1bd4-4961-8530-4c975ddcf643 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:00.213345 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf54381e-1bd4-4961-8530-4c975ddcf643 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:00.214245 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54381e-1bd4-4961-8530-4c975ddcf643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:00:00.217918 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf54381e-1bd4-4961-8530-4c975ddcf643 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:00:00.234015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1270/2520] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 04:00:00 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 04:00:00.238058 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59683568-e465-4bee-a8e3-cdc80b964b9d admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:00:00.253331 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59683568-e465-4bee-a8e3-cdc80b964b9d admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:00:00.262257 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0294ede0-e398-48c5-b0e9-abadcc87c419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:00:00.262951 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294ede0-e398-48c5-b0e9-abadcc87c419 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:00.263406 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294ede0-e398-48c5-b0e9-abadcc87c419 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:00.263682 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294ede0-e398-48c5-b0e9-abadcc87c419 admin admin] PATH_INFO: `/v3/projects/2f05de57855e405cb4ff423353bc85c7/users/de56df051ac546dcb3f5c1d43e348f7e/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:00.266084 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0294ede0-e398-48c5-b0e9-abadcc87c419 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f05de57855e405cb4ff423353bc85c7, user_id=de56df051ac546dcb3f5c1d43e348f7e, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:00:00.269136 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0294ede0-e398-48c5-b0e9-abadcc87c419 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:00:00.281494 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1251/2521] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 04:00:00 2022] PUT /identity/v3/projects/2f05de57855e405cb4ff423353bc85c7/users/de56df051ac546dcb3f5c1d43e348f7e/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 04:00:01.212297 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4347cfe-4042-4625-adc4-f7a2d532acc2 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:01.212827 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4347cfe-4042-4625-adc4-f7a2d532acc2 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:01.213234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4347cfe-4042-4625-adc4-f7a2d532acc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:01.269668 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-c4347cfe-4042-4625-adc4-f7a2d532acc2 None None] MFA Rules not processed for user `1f7a7628896e4005b6e97b02b3266a89`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 04:00:01.271028 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-c4347cfe-4042-4625-adc4-f7a2d532acc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:00:01.431957 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1271/2522] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 04:00:01 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 04:00:01.440922 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0294ede0-e398-48c5-b0e9-abadcc87c419 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:00:01.442004 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-0294ede0-e398-48c5-b0e9-abadcc87c419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:00:01.490423 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:00:01.520118 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0294ede0-e398-48c5-b0e9-abadcc87c419 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:00:01.526534 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f669b-8bb8-48cd-a450-0fefddcba1de service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '565a836c7f89478cad615bd6c14531a7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:00:01.527645 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f669b-8bb8-48cd-a450-0fefddcba1de service glance] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:01.528511 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f669b-8bb8-48cd-a450-0fefddcba1de service glance] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:01.529456 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f669b-8bb8-48cd-a450-0fefddcba1de service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:01.638611 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f669b-8bb8-48cd-a450-0fefddcba1de service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:00:01.640310 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f669b-8bb8-48cd-a450-0fefddcba1de service glance] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:00:01.659139 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1252/2523] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:00:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 17 04:00:02.011779 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba143dc-c9a8-4ffe-9d98-55dbc9528f9f None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:02.012046 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba143dc-c9a8-4ffe-9d98-55dbc9528f9f None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:02.012311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba143dc-c9a8-4ffe-9d98-55dbc9528f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:02.081747 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-2ba143dc-c9a8-4ffe-9d98-55dbc9528f9f None None] MFA Rules not processed for user `cecbc19355ef4570b2b05bba876a3a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 04:00:02.083079 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-2ba143dc-c9a8-4ffe-9d98-55dbc9528f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:00:02.182968 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1272/2524] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Oct 17 04:00:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 04:00:02.384186 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f669b-8bb8-48cd-a450-0fefddcba1de service glance] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:00:02.386018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-166f669b-8bb8-48cd-a450-0fefddcba1de service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:00:02.473869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f669b-8bb8-48cd-a450-0fefddcba1de service glance] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:00:02.481799 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad30799-ac6e-4625-a553-8436db915579 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f64aaf4d408462a88963973d42abfb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:00:02.482772 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad30799-ac6e-4625-a553-8436db915579 service swift] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:02.483158 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad30799-ac6e-4625-a553-8436db915579 service swift] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:02.483509 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad30799-ac6e-4625-a553-8436db915579 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:02.494487 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad30799-ac6e-4625-a553-8436db915579 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:00:02.497677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad30799-ac6e-4625-a553-8436db915579 service swift] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:00:02.509962 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1253/2525] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Oct 17 04:00:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 04:00:08.030663 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013ee1a-1513-47ac-92e3-05fdf4bfad67 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:08.031176 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013ee1a-1513-47ac-92e3-05fdf4bfad67 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:08.031383 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013ee1a-1513-47ac-92e3-05fdf4bfad67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:08.060420 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-1013ee1a-1513-47ac-92e3-05fdf4bfad67 None None] MFA Rules not processed for user `de56df051ac546dcb3f5c1d43e348f7e`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 04:00:08.061647 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-1013ee1a-1513-47ac-92e3-05fdf4bfad67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:00:08.072094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=100 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:00:08.145837 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1273/2526] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 04:00:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 04:00:08.166345 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad30799-ac6e-4625-a553-8436db915579 service swift] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:00:08.168845 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-1ad30799-ac6e-4625-a553-8436db915579 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:00:08.248764 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:00:08.263783 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad30799-ac6e-4625-a553-8436db915579 service swift] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:00:08.271897 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112beef0-0be1-40bf-947d-d8aa96c1b8c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:00:08.272508 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112beef0-0be1-40bf-947d-d8aa96c1b8c2 service nova] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:08.272790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112beef0-0be1-40bf-947d-d8aa96c1b8c2 service nova] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:08.273066 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112beef0-0be1-40bf-947d-d8aa96c1b8c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:08.327641 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112beef0-0be1-40bf-947d-d8aa96c1b8c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:00:08.329268 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112beef0-0be1-40bf-947d-d8aa96c1b8c2 service nova] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:00:08.337921 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1254/2527] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:00:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:00:08.488905 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013ee1a-1513-47ac-92e3-05fdf4bfad67 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:00:08.490600 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-1013ee1a-1513-47ac-92e3-05fdf4bfad67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:00:08.560586 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013ee1a-1513-47ac-92e3-05fdf4bfad67 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:00:08.565052 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6fc2b-ac87-439b-bf95-56f570abf89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:00:08.565728 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6fc2b-ac87-439b-bf95-56f570abf89f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:08.566004 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6fc2b-ac87-439b-bf95-56f570abf89f admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:08.566275 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6fc2b-ac87-439b-bf95-56f570abf89f admin admin] PATH_INFO: `/v3/users/07aaabfabeec47938c3516339682c046` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:08.567392 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe6fc2b-ac87-439b-bf95-56f570abf89f admin admin] RBAC: Authorizing `identity:delete_user(user_id=07aaabfabeec47938c3516339682c046)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:00:08.569984 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe6fc2b-ac87-439b-bf95-56f570abf89f admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:00:08.769070 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2fe6fc2b-ac87-439b-bf95-56f570abf89f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07aaabfabeec47938c3516339682c046'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:00:08.769799 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2fe6fc2b-ac87-439b-bf95-56f570abf89f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07aaabfabeec47938c3516339682c046'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:00:08.848635 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2fe6fc2b-ac87-439b-bf95-56f570abf89f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07aaabfabeec47938c3516339682c046'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:00:08.871175 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2fe6fc2b-ac87-439b-bf95-56f570abf89f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07aaabfabeec47938c3516339682c046'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:00:08.873315 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-2fe6fc2b-ac87-439b-bf95-56f570abf89f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07aaabfabeec47938c3516339682c046'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:00:08.891827 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1274/2528] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 04:00:08 2022] DELETE /identity/v3/users/07aaabfabeec47938c3516339682c046 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 04:00:08.896703 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba9709-ea25-41dc-b67e-cea68d69a2ba None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:08.896920 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba9709-ea25-41dc-b67e-cea68d69a2ba None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:08.897201 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba9709-ea25-41dc-b67e-cea68d69a2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:08.921484 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-afba9709-ea25-41dc-b67e-cea68d69a2ba None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 04:00:08.922993 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-afba9709-ea25-41dc-b67e-cea68d69a2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:00:08.998046 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1255/2529] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 04:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 04:00:09.018619 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6fc2b-ac87-439b-bf95-56f570abf89f admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:00:09.020116 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-2fe6fc2b-ac87-439b-bf95-56f570abf89f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:00:09.074864 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:00:09.144560 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6fc2b-ac87-439b-bf95-56f570abf89f admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:00:09.154261 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484ed02-3c21-496a-869e-4290b39ec136 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:00:09.156086 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484ed02-3c21-496a-869e-4290b39ec136 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:09.160862 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484ed02-3c21-496a-869e-4290b39ec136 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:09.161405 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484ed02-3c21-496a-869e-4290b39ec136 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:09.174617 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a484ed02-3c21-496a-869e-4290b39ec136 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:00:09.179278 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a484ed02-3c21-496a-869e-4290b39ec136 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:00:09.193353 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1275/2530] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:00:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:00:09.755333 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba9709-ea25-41dc-b67e-cea68d69a2ba None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:00:09.764319 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-afba9709-ea25-41dc-b67e-cea68d69a2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:00:09.788162 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba9709-ea25-41dc-b67e-cea68d69a2ba None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:00:09.795377 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1518ec4a-3748-419a-a8e2-1bc018a36b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:00:09.796641 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518ec4a-3748-419a-a8e2-1bc018a36b1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:09.798050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518ec4a-3748-419a-a8e2-1bc018a36b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:09.798050 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1518ec4a-3748-419a-a8e2-1bc018a36b1d admin admin] PATH_INFO: `/v3/projects/87537590ed3b449b91be75cf1811e527` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:09.799961 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1518ec4a-3748-419a-a8e2-1bc018a36b1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=87537590ed3b449b91be75cf1811e527)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:00:09.802123 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1518ec4a-3748-419a-a8e2-1bc018a36b1d admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:00:09.963786 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-1518ec4a-3748-419a-a8e2-1bc018a36b1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87537590ed3b449b91be75cf1811e527', 'request_id': 'req-1518ec4a-3748-419a-a8e2-1bc018a36b1d'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:00:10.037113 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-1518ec4a-3748-419a-a8e2-1bc018a36b1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87537590ed3b449b91be75cf1811e527', 'request_id': 'req-1518ec4a-3748-419a-a8e2-1bc018a36b1d'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:00:10.046900 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-1518ec4a-3748-419a-a8e2-1bc018a36b1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87537590ed3b449b91be75cf1811e527', 'request_id': 'req-1518ec4a-3748-419a-a8e2-1bc018a36b1d'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:00:10.059754 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-1518ec4a-3748-419a-a8e2-1bc018a36b1d admin admin] The token cache is being invalidate because project 87537590ed3b449b91be75cf1811e527 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 04:00:10.060075 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-1518ec4a-3748-419a-a8e2-1bc018a36b1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:00:10.060756 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-1518ec4a-3748-419a-a8e2-1bc018a36b1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:00:10.065663 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1256/2531] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 04:00:09 2022] DELETE /identity/v3/projects/87537590ed3b449b91be75cf1811e527 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 04:00:10.067418 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa13d7-48f4-46c9-aa2f-a860dfd1efdb None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:10.068115 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa13d7-48f4-46c9-aa2f-a860dfd1efdb None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:10.068115 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aa13d7-48f4-46c9-aa2f-a860dfd1efdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:10.125860 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-84aa13d7-48f4-46c9-aa2f-a860dfd1efdb None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 04:00:10.127332 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-84aa13d7-48f4-46c9-aa2f-a860dfd1efdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:00:10.241616 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1276/2532] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 04:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 04:00:10.250750 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a509e5f-022e-4fb2-a9a5-a9b5dd6b5dad None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:10.253347 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a509e5f-022e-4fb2-a9a5-a9b5dd6b5dad None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:10.253347 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a509e5f-022e-4fb2-a9a5-a9b5dd6b5dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:10.278445 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-3a509e5f-022e-4fb2-a9a5-a9b5dd6b5dad None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 04:00:10.279897 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-3a509e5f-022e-4fb2-a9a5-a9b5dd6b5dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:00:10.299805 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1257/2533] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 04:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 04:00:10.301906 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa13d7-48f4-46c9-aa2f-a860dfd1efdb None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:00:10.322542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aa13d7-48f4-46c9-aa2f-a860dfd1efdb None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:00:10.338518 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23888e15-ca3c-461f-901c-5b31e3a25952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:00:10.339848 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23888e15-ca3c-461f-901c-5b31e3a25952 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:10.340396 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23888e15-ca3c-461f-901c-5b31e3a25952 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:10.340885 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23888e15-ca3c-461f-901c-5b31e3a25952 admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:10.344720 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23888e15-ca3c-461f-901c-5b31e3a25952 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 04:00:10.345320 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23888e15-ca3c-461f-901c-5b31e3a25952 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:00:10.350628 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23888e15-ca3c-461f-901c-5b31e3a25952 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:00:10.372191 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a509e5f-022e-4fb2-a9a5-a9b5dd6b5dad None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:00:10.377672 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1277/2534] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 04:00:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 04:00:10.383106 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a509e5f-022e-4fb2-a9a5-a9b5dd6b5dad None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:00:10.388340 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fab4c7-0792-49b4-bfe9-b6a184789a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:00:10.389170 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fab4c7-0792-49b4-bfe9-b6a184789a0f admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:10.389643 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fab4c7-0792-49b4-bfe9-b6a184789a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:10.390117 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fab4c7-0792-49b4-bfe9-b6a184789a0f admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:10.390864 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fab4c7-0792-49b4-bfe9-b6a184789a0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:00:10.392915 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fab4c7-0792-49b4-bfe9-b6a184789a0f admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:00:10.399801 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:00:10.507841 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1258/2535] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 04:00:10 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 04:00:10.512712 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23888e15-ca3c-461f-901c-5b31e3a25952 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:00:10.535301 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23888e15-ca3c-461f-901c-5b31e3a25952 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:00:10.643815 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1338be-cfa2-4a69-90fe-ae1d09b8b3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:00:10.644447 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1338be-cfa2-4a69-90fe-ae1d09b8b3a8 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:10.646226 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1338be-cfa2-4a69-90fe-ae1d09b8b3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:10.646226 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1338be-cfa2-4a69-90fe-ae1d09b8b3a8 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:10.646226 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1338be-cfa2-4a69-90fe-ae1d09b8b3a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:00:10.648124 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1338be-cfa2-4a69-90fe-ae1d09b8b3a8 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:00:10.735854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1278/2536] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 04:00:10 2022] POST /identity/v3/users => generated 287 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 04:00:10.738882 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fab4c7-0792-49b4-bfe9-b6a184789a0f admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:00:10.750017 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fab4c7-0792-49b4-bfe9-b6a184789a0f admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:00:10.755580 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b7e01-185a-405b-a06e-874193449289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:00:10.756995 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b7e01-185a-405b-a06e-874193449289 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:10.757277 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b7e01-185a-405b-a06e-874193449289 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:10.757547 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51b7e01-185a-405b-a06e-874193449289 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:10.758285 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b7e01-185a-405b-a06e-874193449289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:00:10.760011 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51b7e01-185a-405b-a06e-874193449289 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:00:10.777561 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1259/2537] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 04:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 04:00:10.781820 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1338be-cfa2-4a69-90fe-ae1d09b8b3a8 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:00:10.796323 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1338be-cfa2-4a69-90fe-ae1d09b8b3a8 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:00:10.802015 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6c737-934d-4c3e-a914-1258540627a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:00:10.809916 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6c737-934d-4c3e-a914-1258540627a8 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:10.810262 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6c737-934d-4c3e-a914-1258540627a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:10.810556 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6c737-934d-4c3e-a914-1258540627a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:10.811528 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d6c737-934d-4c3e-a914-1258540627a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:00:10.813258 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d6c737-934d-4c3e-a914-1258540627a8 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:00:10.830390 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1279/2538] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 04:00:10 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 17 04:00:10.837677 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b7e01-185a-405b-a06e-874193449289 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:00:10.849780 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51b7e01-185a-405b-a06e-874193449289 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:00:10.856725 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c7867-468f-4e95-a7df-9c3ac4ea3f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:00:10.857751 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c7867-468f-4e95-a7df-9c3ac4ea3f81 admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:10.858233 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c7867-468f-4e95-a7df-9c3ac4ea3f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:10.858785 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c7867-468f-4e95-a7df-9c3ac4ea3f81 admin admin] PATH_INFO: `/v3/projects/7df5c39d80d8467c849ec8eef3a62f75/users/65b393a3e85d4fb0b8e28ed0b6e63299/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:10.889425 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c7867-468f-4e95-a7df-9c3ac4ea3f81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7df5c39d80d8467c849ec8eef3a62f75, user_id=65b393a3e85d4fb0b8e28ed0b6e63299, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:00:10.892648 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c7867-468f-4e95-a7df-9c3ac4ea3f81 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:00:10.993241 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1260/2539] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 04:00:10 2022] PUT /identity/v3/projects/7df5c39d80d8467c849ec8eef3a62f75/users/65b393a3e85d4fb0b8e28ed0b6e63299/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 04:00:11.003428 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9a861-6a03-4b88-8cc6-65ae533ee6b4 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:11.004174 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9a861-6a03-4b88-8cc6-65ae533ee6b4 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:11.004551 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9a861-6a03-4b88-8cc6-65ae533ee6b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:11.049407 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-eff9a861-6a03-4b88-8cc6-65ae533ee6b4 None None] MFA Rules not processed for user `65b393a3e85d4fb0b8e28ed0b6e63299`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 04:00:11.051291 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-eff9a861-6a03-4b88-8cc6-65ae533ee6b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:00:11.120279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:00:11.198749 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1280/2540] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 04:00:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 04:00:11.210098 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c7867-468f-4e95-a7df-9c3ac4ea3f81 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:00:11.212100 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-8d1c7867-468f-4e95-a7df-9c3ac4ea3f81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:00:11.307513 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c7867-468f-4e95-a7df-9c3ac4ea3f81 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:00:11.313190 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e85b8-ad65-45e4-888e-a3af14733d43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99bc94105bbf4cb4a60e37133edc8dfb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:00:11.314419 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e85b8-ad65-45e4-888e-a3af14733d43 service cinder] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:11.314829 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e85b8-ad65-45e4-888e-a3af14733d43 service cinder] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:11.315231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e85b8-ad65-45e4-888e-a3af14733d43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:11.326144 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e85b8-ad65-45e4-888e-a3af14733d43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:00:11.329156 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e85b8-ad65-45e4-888e-a3af14733d43 service cinder] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:00:11.345940 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1261/2541] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:00:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:00:11.773492 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b608824-8c32-497a-8591-c3f19345ae0a None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:11.773492 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b608824-8c32-497a-8591-c3f19345ae0a None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:11.773492 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b608824-8c32-497a-8591-c3f19345ae0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:11.801776 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-1b608824-8c32-497a-8591-c3f19345ae0a None None] MFA Rules not processed for user `926a7f570adb46debe57caefead0e17b`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 04:00:11.803546 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-1b608824-8c32-497a-8591-c3f19345ae0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:00:11.884857 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1281/2542] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 04:00:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 04:00:16.732052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9b562-5e64-448e-ab89-71fd43476dd0 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:16.732654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9b562-5e64-448e-ab89-71fd43476dd0 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:16.732654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9b562-5e64-448e-ab89-71fd43476dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:16.752370 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-80c9b562-5e64-448e-ab89-71fd43476dd0 None None] MFA Rules not processed for user `b86807317d1247c4a9afdf4c1afa3a2d`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 04:00:16.753644 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-80c9b562-5e64-448e-ab89-71fd43476dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:00:16.814574 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1262/2543] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 04:00:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 04:00:20.276341 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb795b7-5161-4832-97f0-acebf11d2859 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:20.276873 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb795b7-5161-4832-97f0-acebf11d2859 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:20.277059 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb795b7-5161-4832-97f0-acebf11d2859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:20.297973 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-abb795b7-5161-4832-97f0-acebf11d2859 None None] MFA Rules not processed for user `cecbc19355ef4570b2b05bba876a3a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 04:00:20.299304 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-abb795b7-5161-4832-97f0-acebf11d2859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:00:20.366433 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1282/2544] 10.222.0.17 () {58 vars in 1009 bytes} [Mon Oct 17 04:00:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 04:00:20.379303 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9b562-5e64-448e-ab89-71fd43476dd0 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:00:20.381428 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-80c9b562-5e64-448e-ab89-71fd43476dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:00:20.451652 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:00:20.457871 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9b562-5e64-448e-ab89-71fd43476dd0 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:00:20.463833 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ba0592-0831-4879-b180-f74ab05668ae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f64aaf4d408462a88963973d42abfb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:00:20.464708 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba0592-0831-4879-b180-f74ab05668ae service swift] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:20.464987 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba0592-0831-4879-b180-f74ab05668ae service swift] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:20.465258 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ba0592-0831-4879-b180-f74ab05668ae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:20.474230 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ba0592-0831-4879-b180-f74ab05668ae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:00:20.475974 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ba0592-0831-4879-b180-f74ab05668ae service swift] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:00:20.484516 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1263/2545] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Oct 17 04:00:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 04:00:23.690736 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb795b7-5161-4832-97f0-acebf11d2859 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:00:23.768152 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb795b7-5161-4832-97f0-acebf11d2859 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:00:23.773088 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee17a409-421f-4495-8bc4-1855fdb9e073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:00:23.773937 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee17a409-421f-4495-8bc4-1855fdb9e073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:23.774222 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee17a409-421f-4495-8bc4-1855fdb9e073 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:23.774495 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee17a409-421f-4495-8bc4-1855fdb9e073 admin admin] PATH_INFO: `/v3/users/65b393a3e85d4fb0b8e28ed0b6e63299` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:23.775893 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee17a409-421f-4495-8bc4-1855fdb9e073 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65b393a3e85d4fb0b8e28ed0b6e63299)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:00:23.778081 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee17a409-421f-4495-8bc4-1855fdb9e073 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:00:23.797221 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:00:23.982928 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ee17a409-421f-4495-8bc4-1855fdb9e073 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65b393a3e85d4fb0b8e28ed0b6e63299'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:00:23.984096 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ee17a409-421f-4495-8bc4-1855fdb9e073 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65b393a3e85d4fb0b8e28ed0b6e63299'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:00:24.064024 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ee17a409-421f-4495-8bc4-1855fdb9e073 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65b393a3e85d4fb0b8e28ed0b6e63299'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:00:24.078286 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ee17a409-421f-4495-8bc4-1855fdb9e073 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65b393a3e85d4fb0b8e28ed0b6e63299'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:00:24.079109 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-ee17a409-421f-4495-8bc4-1855fdb9e073 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65b393a3e85d4fb0b8e28ed0b6e63299'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:00:24.101576 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1283/2546] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 04:00:23 2022] DELETE /identity/v3/users/65b393a3e85d4fb0b8e28ed0b6e63299 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 04:00:24.106985 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a65ee7-dd79-41a4-9c78-1a9e24c81577 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:24.108176 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a65ee7-dd79-41a4-9c78-1a9e24c81577 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:24.108763 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a65ee7-dd79-41a4-9c78-1a9e24c81577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:24.131557 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-62a65ee7-dd79-41a4-9c78-1a9e24c81577 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 04:00:24.132800 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-62a65ee7-dd79-41a4-9c78-1a9e24c81577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:00:24.209485 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1264/2547] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 04:00:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 17 04:00:24.225069 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee17a409-421f-4495-8bc4-1855fdb9e073 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:00:24.225069 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-ee17a409-421f-4495-8bc4-1855fdb9e073 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:00:24.289203 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee17a409-421f-4495-8bc4-1855fdb9e073 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:00:24.293035 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db169713-dec3-47ab-bfa9-31f8937a8815 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:00:24.293651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db169713-dec3-47ab-bfa9-31f8937a8815 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:24.294008 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db169713-dec3-47ab-bfa9-31f8937a8815 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:24.294285 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db169713-dec3-47ab-bfa9-31f8937a8815 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:24.304494 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db169713-dec3-47ab-bfa9-31f8937a8815 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:00:24.306094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db169713-dec3-47ab-bfa9-31f8937a8815 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:00:24.316595 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1284/2548] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:00:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:00:24.754426 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a65ee7-dd79-41a4-9c78-1a9e24c81577 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:00:24.756935 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-62a65ee7-dd79-41a4-9c78-1a9e24c81577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:00:24.777173 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a65ee7-dd79-41a4-9c78-1a9e24c81577 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:00:24.795491 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71813d63-10c8-4b8e-bcd4-3b95933ab67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:00:24.796200 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71813d63-10c8-4b8e-bcd4-3b95933ab67e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:24.796487 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71813d63-10c8-4b8e-bcd4-3b95933ab67e admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:24.796768 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71813d63-10c8-4b8e-bcd4-3b95933ab67e admin admin] PATH_INFO: `/v3/projects/7df5c39d80d8467c849ec8eef3a62f75` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:24.797961 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71813d63-10c8-4b8e-bcd4-3b95933ab67e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7df5c39d80d8467c849ec8eef3a62f75)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:00:24.799912 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71813d63-10c8-4b8e-bcd4-3b95933ab67e admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:00:24.940624 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-71813d63-10c8-4b8e-bcd4-3b95933ab67e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7df5c39d80d8467c849ec8eef3a62f75', 'request_id': 'req-71813d63-10c8-4b8e-bcd4-3b95933ab67e'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:00:25.068223 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-71813d63-10c8-4b8e-bcd4-3b95933ab67e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7df5c39d80d8467c849ec8eef3a62f75', 'request_id': 'req-71813d63-10c8-4b8e-bcd4-3b95933ab67e'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:00:25.075440 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-71813d63-10c8-4b8e-bcd4-3b95933ab67e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7df5c39d80d8467c849ec8eef3a62f75', 'request_id': 'req-71813d63-10c8-4b8e-bcd4-3b95933ab67e'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:00:25.089627 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-71813d63-10c8-4b8e-bcd4-3b95933ab67e admin admin] The token cache is being invalidate because project 7df5c39d80d8467c849ec8eef3a62f75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 04:00:25.090073 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-71813d63-10c8-4b8e-bcd4-3b95933ab67e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:00:25.090749 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-71813d63-10c8-4b8e-bcd4-3b95933ab67e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:00:25.102734 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1265/2549] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 04:00:24 2022] DELETE /identity/v3/projects/7df5c39d80d8467c849ec8eef3a62f75 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 04:00:34.314222 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:DELETE=9,keystone:SELECT=46,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:00:35.085682 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=59,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:00:42.719238 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db169713-dec3-47ab-bfa9-31f8937a8815 service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:00:42.728444 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db169713-dec3-47ab-bfa9-31f8937a8815 service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:00:42.776392 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d278c-9b6f-4c86-8717-67c7ae2254ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:00:42.776988 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d278c-9b6f-4c86-8717-67c7ae2254ca service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:42.777265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d278c-9b6f-4c86-8717-67c7ae2254ca service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:42.777539 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d278c-9b6f-4c86-8717-67c7ae2254ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:42.779279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-ec7d278c-9b6f-4c86-8717-67c7ae2254ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:00:42.848954 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d278c-9b6f-4c86-8717-67c7ae2254ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:00:42.850977 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d278c-9b6f-4c86-8717-67c7ae2254ca service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:00:42.862090 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1285/2550] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:00:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:00:44.038472 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71813d63-10c8-4b8e-bcd4-3b95933ab67e admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:00:44.039937 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-71813d63-10c8-4b8e-bcd4-3b95933ab67e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:00:44.052134 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71813d63-10c8-4b8e-bcd4-3b95933ab67e admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:00:44.056375 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01da5ef-46b8-4028-80f1-04cebb73cebd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:00:44.056976 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01da5ef-46b8-4028-80f1-04cebb73cebd service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:44.057257 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01da5ef-46b8-4028-80f1-04cebb73cebd service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:44.057530 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01da5ef-46b8-4028-80f1-04cebb73cebd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:44.059259 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-f01da5ef-46b8-4028-80f1-04cebb73cebd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:00:44.072369 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01da5ef-46b8-4028-80f1-04cebb73cebd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:00:44.073800 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01da5ef-46b8-4028-80f1-04cebb73cebd service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:00:44.084490 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1266/2551] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:00:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:00:46.647697 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d278c-9b6f-4c86-8717-67c7ae2254ca service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:00:46.656459 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d278c-9b6f-4c86-8717-67c7ae2254ca service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:00:46.660211 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc147d1-f6d0-4954-911d-d845ac24105f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:00:46.660867 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc147d1-f6d0-4954-911d-d845ac24105f service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:46.661145 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc147d1-f6d0-4954-911d-d845ac24105f service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:46.661417 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc147d1-f6d0-4954-911d-d845ac24105f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:46.663158 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-4cc147d1-f6d0-4954-911d-d845ac24105f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:00:46.727283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc147d1-f6d0-4954-911d-d845ac24105f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:00:46.728739 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc147d1-f6d0-4954-911d-d845ac24105f service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:00:46.738021 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1286/2552] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:00:46 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:00:47.997970 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01da5ef-46b8-4028-80f1-04cebb73cebd service neutron] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:00:48.007597 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01da5ef-46b8-4028-80f1-04cebb73cebd service neutron] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:00:48.011267 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571697d4-70da-4cb3-9110-b78ba00b10bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:00:48.011904 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571697d4-70da-4cb3-9110-b78ba00b10bf service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:48.012181 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571697d4-70da-4cb3-9110-b78ba00b10bf service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:48.012447 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571697d4-70da-4cb3-9110-b78ba00b10bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:48.014155 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-571697d4-70da-4cb3-9110-b78ba00b10bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:00:48.078992 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571697d4-70da-4cb3-9110-b78ba00b10bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:00:48.080347 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571697d4-70da-4cb3-9110-b78ba00b10bf service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:00:48.091057 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1267/2553] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:00:47 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:00:48.919107 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc147d1-f6d0-4954-911d-d845ac24105f service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:00:48.920817 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-4cc147d1-f6d0-4954-911d-d845ac24105f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:00:48.936959 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc147d1-f6d0-4954-911d-d845ac24105f service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:00:48.942957 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36e4bf-30d5-410e-abd9-d79da4b13e86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:00:48.944283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36e4bf-30d5-410e-abd9-d79da4b13e86 service nova] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:00:48.944677 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36e4bf-30d5-410e-abd9-d79da4b13e86 service nova] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:00:48.945054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36e4bf-30d5-410e-abd9-d79da4b13e86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:00:48.948039 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-cd36e4bf-30d5-410e-abd9-d79da4b13e86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:00:48.960861 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36e4bf-30d5-410e-abd9-d79da4b13e86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:00:48.962590 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36e4bf-30d5-410e-abd9-d79da4b13e86 service nova] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:00:48.975021 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1287/2554] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:00:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:00:58.086774 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=36 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:00:58.969264 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=90 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:01:15.596968 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571697d4-70da-4cb3-9110-b78ba00b10bf service neutron] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:01:15.606326 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571697d4-70da-4cb3-9110-b78ba00b10bf service neutron] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:01:15.610012 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a93d53-11db-4d8f-9a88-af3173399d8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:01:15.610748 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a93d53-11db-4d8f-9a88-af3173399d8d service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:01:15.611029 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a93d53-11db-4d8f-9a88-af3173399d8d service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:01:15.611297 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a93d53-11db-4d8f-9a88-af3173399d8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:01:15.613026 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-c0a93d53-11db-4d8f-9a88-af3173399d8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:01:15.623003 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a93d53-11db-4d8f-9a88-af3173399d8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:01:15.624473 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a93d53-11db-4d8f-9a88-af3173399d8d service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:01:15.633306 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1268/2555] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:01:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:01:25.628987 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=6 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:02:14.278013 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36e4bf-30d5-410e-abd9-d79da4b13e86 service nova] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:02:14.287847 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-cd36e4bf-30d5-410e-abd9-d79da4b13e86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:02:14.294024 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36e4bf-30d5-410e-abd9-d79da4b13e86 service nova] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:02:14.300273 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce22aa-0fb4-4130-90b0-2e14127da97e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:02:14.300865 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce22aa-0fb4-4130-90b0-2e14127da97e service nova] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:14.301140 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce22aa-0fb4-4130-90b0-2e14127da97e service nova] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:14.301409 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ce22aa-0fb4-4130-90b0-2e14127da97e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:14.303097 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-c6ce22aa-0fb4-4130-90b0-2e14127da97e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:02:14.314916 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ce22aa-0fb4-4130-90b0-2e14127da97e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:02:14.316508 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ce22aa-0fb4-4130-90b0-2e14127da97e service nova] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:02:14.326904 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1288/2556] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:02:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:02:16.461715 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375ced7-f5d1-48e4-805e-239ec83c74db None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:16.462181 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375ced7-f5d1-48e4-805e-239ec83c74db None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:16.462294 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375ced7-f5d1-48e4-805e-239ec83c74db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:16.511587 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-e375ced7-f5d1-48e4-805e-239ec83c74db None None] MFA Rules not processed for user `cecbc19355ef4570b2b05bba876a3a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 04:02:16.512862 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-e375ced7-f5d1-48e4-805e-239ec83c74db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:02:16.575721 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1269/2557] 10.222.0.17 () {58 vars in 967 bytes} [Mon Oct 17 04:02:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 04:02:16.616731 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce22aa-0fb4-4130-90b0-2e14127da97e service nova] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:02:16.618134 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-c6ce22aa-0fb4-4130-90b0-2e14127da97e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:02:16.710017 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ce22aa-0fb4-4130-90b0-2e14127da97e service nova] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:02:16.715514 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478b07c-16be-454c-b53a-ba7dcb1526f1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f64aaf4d408462a88963973d42abfb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:02:16.716460 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478b07c-16be-454c-b53a-ba7dcb1526f1 service swift] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:16.716744 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478b07c-16be-454c-b53a-ba7dcb1526f1 service swift] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:16.717021 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478b07c-16be-454c-b53a-ba7dcb1526f1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:16.725609 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c478b07c-16be-454c-b53a-ba7dcb1526f1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:02:16.727279 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c478b07c-16be-454c-b53a-ba7dcb1526f1 service swift] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:02:16.735757 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1289/2558] 10.222.0.17 () {60 vars in 1384 bytes} [Mon Oct 17 04:02:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 17 04:02:20.664803 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375ced7-f5d1-48e4-805e-239ec83c74db None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:02:20.666440 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-e375ced7-f5d1-48e4-805e-239ec83c74db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:02:20.745390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375ced7-f5d1-48e4-805e-239ec83c74db None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:02:20.750313 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7fb9f-5006-4875-803a-f50de63f8933 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b500a113a2aa4a95875d7e094b917db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:02:20.750919 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7fb9f-5006-4875-803a-f50de63f8933 service placement] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:20.751196 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7fb9f-5006-4875-803a-f50de63f8933 service placement] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:20.751466 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c7fb9f-5006-4875-803a-f50de63f8933 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:20.753224 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-29c7fb9f-5006-4875-803a-f50de63f8933 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:02:20.765427 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c7fb9f-5006-4875-803a-f50de63f8933 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:02:20.767568 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c7fb9f-5006-4875-803a-f50de63f8933 service placement] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:02:20.779196 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1270/2559] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:02:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:02:24.302920 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478b07c-16be-454c-b53a-ba7dcb1526f1 service swift] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:02:24.304408 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-c478b07c-16be-454c-b53a-ba7dcb1526f1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:02:24.375776 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478b07c-16be-454c-b53a-ba7dcb1526f1 service swift] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:02:24.380244 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b4ec2-b1b0-4bad-8de7-57f5b5499ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:02:24.380869 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b4ec2-b1b0-4bad-8de7-57f5b5499ece admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:24.381155 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b4ec2-b1b0-4bad-8de7-57f5b5499ece admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:24.381429 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889b4ec2-b1b0-4bad-8de7-57f5b5499ece admin admin] PATH_INFO: `/v3/users/b86807317d1247c4a9afdf4c1afa3a2d` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:24.382541 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b4ec2-b1b0-4bad-8de7-57f5b5499ece admin admin] RBAC: Authorizing `identity:delete_user(user_id=b86807317d1247c4a9afdf4c1afa3a2d)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:02:24.384477 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889b4ec2-b1b0-4bad-8de7-57f5b5499ece admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:02:24.538824 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-889b4ec2-b1b0-4bad-8de7-57f5b5499ece admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b86807317d1247c4a9afdf4c1afa3a2d'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:02:24.539492 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-889b4ec2-b1b0-4bad-8de7-57f5b5499ece admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b86807317d1247c4a9afdf4c1afa3a2d'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:02:24.581953 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-889b4ec2-b1b0-4bad-8de7-57f5b5499ece admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b86807317d1247c4a9afdf4c1afa3a2d'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:02:24.586468 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:02:24.595570 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-889b4ec2-b1b0-4bad-8de7-57f5b5499ece admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b86807317d1247c4a9afdf4c1afa3a2d'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:02:24.596234 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-889b4ec2-b1b0-4bad-8de7-57f5b5499ece admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b86807317d1247c4a9afdf4c1afa3a2d'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:02:24.608238 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1290/2560] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 04:02:24 2022] DELETE /identity/v3/users/b86807317d1247c4a9afdf4c1afa3a2d => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 04:02:24.838520 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7fb9f-5006-4875-803a-f50de63f8933 service placement] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:02:24.848705 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c7fb9f-5006-4875-803a-f50de63f8933 service placement] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:02:24.905836 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea700754-1cef-4c05-bd60-32d39905198f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:02:24.906447 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea700754-1cef-4c05-bd60-32d39905198f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:24.906730 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea700754-1cef-4c05-bd60-32d39905198f admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:24.907006 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea700754-1cef-4c05-bd60-32d39905198f admin admin] PATH_INFO: `/v3/projects/f1fea935de384bfa81953ec336e06b7f` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:24.908151 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea700754-1cef-4c05-bd60-32d39905198f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1fea935de384bfa81953ec336e06b7f)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:02:24.910083 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea700754-1cef-4c05-bd60-32d39905198f admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:02:24.912703 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=100 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:02:25.017711 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-ea700754-1cef-4c05-bd60-32d39905198f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1fea935de384bfa81953ec336e06b7f', 'request_id': 'req-ea700754-1cef-4c05-bd60-32d39905198f'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:02:25.096805 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-ea700754-1cef-4c05-bd60-32d39905198f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1fea935de384bfa81953ec336e06b7f', 'request_id': 'req-ea700754-1cef-4c05-bd60-32d39905198f'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:02:25.102720 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-ea700754-1cef-4c05-bd60-32d39905198f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1fea935de384bfa81953ec336e06b7f', 'request_id': 'req-ea700754-1cef-4c05-bd60-32d39905198f'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:02:25.114772 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-ea700754-1cef-4c05-bd60-32d39905198f admin admin] The token cache is being invalidate because project f1fea935de384bfa81953ec336e06b7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 04:02:25.115060 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-ea700754-1cef-4c05-bd60-32d39905198f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:02:25.115678 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-ea700754-1cef-4c05-bd60-32d39905198f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:02:25.117177 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1271/2561] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 04:02:24 2022] DELETE /identity/v3/projects/f1fea935de384bfa81953ec336e06b7f => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 04:02:25.121219 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b4ec2-b1b0-4bad-8de7-57f5b5499ece admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:02:25.122613 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-889b4ec2-b1b0-4bad-8de7-57f5b5499ece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:02:25.136321 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889b4ec2-b1b0-4bad-8de7-57f5b5499ece admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:02:25.191871 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b1022e-2aa4-473c-9c15-b14311b40bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:02:25.192506 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b1022e-2aa4-473c-9c15-b14311b40bed admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:25.192792 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b1022e-2aa4-473c-9c15-b14311b40bed admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:25.193071 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b1022e-2aa4-473c-9c15-b14311b40bed admin admin] PATH_INFO: `/v3/users/1f7a7628896e4005b6e97b02b3266a89` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:25.194193 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b1022e-2aa4-473c-9c15-b14311b40bed admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f7a7628896e4005b6e97b02b3266a89)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:02:25.196214 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b1022e-2aa4-473c-9c15-b14311b40bed admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:02:25.327127 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-94b1022e-2aa4-473c-9c15-b14311b40bed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f7a7628896e4005b6e97b02b3266a89'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:02:25.327824 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-94b1022e-2aa4-473c-9c15-b14311b40bed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f7a7628896e4005b6e97b02b3266a89'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:02:25.382216 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-94b1022e-2aa4-473c-9c15-b14311b40bed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f7a7628896e4005b6e97b02b3266a89'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:02:25.394477 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-94b1022e-2aa4-473c-9c15-b14311b40bed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f7a7628896e4005b6e97b02b3266a89'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:02:25.398558 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-94b1022e-2aa4-473c-9c15-b14311b40bed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f7a7628896e4005b6e97b02b3266a89'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:02:25.407430 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1291/2562] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 04:02:25 2022] DELETE /identity/v3/users/1f7a7628896e4005b6e97b02b3266a89 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 04:02:25.640679 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea700754-1cef-4c05-bd60-32d39905198f admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:02:25.641842 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-ea700754-1cef-4c05-bd60-32d39905198f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:02:25.658371 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea700754-1cef-4c05-bd60-32d39905198f admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:02:25.715336 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a482b5-efec-42da-931d-44af0869c274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:02:25.716247 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a482b5-efec-42da-931d-44af0869c274 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:25.716524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a482b5-efec-42da-931d-44af0869c274 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:25.716792 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a482b5-efec-42da-931d-44af0869c274 admin admin] PATH_INFO: `/v3/projects/ccc8c4e5d2864b9cb530c524185c792e` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:25.717895 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a482b5-efec-42da-931d-44af0869c274 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccc8c4e5d2864b9cb530c524185c792e)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:02:25.719790 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a482b5-efec-42da-931d-44af0869c274 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:02:25.833399 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-77a482b5-efec-42da-931d-44af0869c274 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccc8c4e5d2864b9cb530c524185c792e', 'request_id': 'req-77a482b5-efec-42da-931d-44af0869c274'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:02:25.894907 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-77a482b5-efec-42da-931d-44af0869c274 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccc8c4e5d2864b9cb530c524185c792e', 'request_id': 'req-77a482b5-efec-42da-931d-44af0869c274'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:02:25.900504 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-77a482b5-efec-42da-931d-44af0869c274 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccc8c4e5d2864b9cb530c524185c792e', 'request_id': 'req-77a482b5-efec-42da-931d-44af0869c274'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:02:25.911783 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-77a482b5-efec-42da-931d-44af0869c274 admin admin] The token cache is being invalidate because project ccc8c4e5d2864b9cb530c524185c792e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 04:02:25.912113 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-77a482b5-efec-42da-931d-44af0869c274 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:02:25.912797 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-77a482b5-efec-42da-931d-44af0869c274 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:02:25.914258 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1272/2563] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 04:02:25 2022] DELETE /identity/v3/projects/ccc8c4e5d2864b9cb530c524185c792e => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 04:02:25.919163 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f8004-7c09-463a-a846-04bf820d3c49 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:25.919453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f8004-7c09-463a-a846-04bf820d3c49 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:25.919746 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12f8004-7c09-463a-a846-04bf820d3c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:25.940323 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-f12f8004-7c09-463a-a846-04bf820d3c49 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 04:02:25.941536 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-f12f8004-7c09-463a-a846-04bf820d3c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:02:25.988470 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:02:26.000340 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1292/2564] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 04:02:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 04:02:26.008250 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c32ae-3931-4a72-a2f8-3ed602980ef0 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:26.008538 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c32ae-3931-4a72-a2f8-3ed602980ef0 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:26.008810 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944c32ae-3931-4a72-a2f8-3ed602980ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:26.028378 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-944c32ae-3931-4a72-a2f8-3ed602980ef0 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 04:02:26.029579 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-944c32ae-3931-4a72-a2f8-3ed602980ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:02:26.037519 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1273/2565] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 04:02:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 04:02:26.041658 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f8004-7c09-463a-a846-04bf820d3c49 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:02:26.052606 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12f8004-7c09-463a-a846-04bf820d3c49 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:02:26.057282 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eda931-a899-4fed-8907-1e8a1ce1727d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:02:26.057889 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eda931-a899-4fed-8907-1e8a1ce1727d admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:26.058168 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eda931-a899-4fed-8907-1e8a1ce1727d admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:26.058442 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eda931-a899-4fed-8907-1e8a1ce1727d admin admin] PATH_INFO: `/v3/domains` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:26.059090 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eda931-a899-4fed-8907-1e8a1ce1727d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88299) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 17 04:02:26.059395 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eda931-a899-4fed-8907-1e8a1ce1727d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:02:26.061108 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eda931-a899-4fed-8907-1e8a1ce1727d admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:02:26.073376 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1293/2566] 10.222.0.17 () {58 vars in 1153 bytes} [Mon Oct 17 04:02:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 17 04:02:26.077480 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c32ae-3931-4a72-a2f8-3ed602980ef0 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:02:26.086481 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944c32ae-3931-4a72-a2f8-3ed602980ef0 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:02:26.091390 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbcb5df-edc0-47c4-8b7b-83c685353c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:02:26.092018 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbcb5df-edc0-47c4-8b7b-83c685353c5a admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:26.092300 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbcb5df-edc0-47c4-8b7b-83c685353c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:26.092570 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbcb5df-edc0-47c4-8b7b-83c685353c5a admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:26.093223 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbcb5df-edc0-47c4-8b7b-83c685353c5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:02:26.095100 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbcb5df-edc0-47c4-8b7b-83c685353c5a admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:02:26.109848 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:02:26.200182 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1274/2567] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 04:02:26 2022] POST /identity/v3/projects => generated 379 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 04:02:26.204451 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eda931-a899-4fed-8907-1e8a1ce1727d admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:02:26.214334 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eda931-a899-4fed-8907-1e8a1ce1727d admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:02:26.269794 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf5ed6-ffde-4c53-859b-fec92959d6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:02:26.270460 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf5ed6-ffde-4c53-859b-fec92959d6b7 admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:26.270739 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf5ed6-ffde-4c53-859b-fec92959d6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:26.271009 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf5ed6-ffde-4c53-859b-fec92959d6b7 admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:26.271709 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf5ed6-ffde-4c53-859b-fec92959d6b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:02:26.273669 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf5ed6-ffde-4c53-859b-fec92959d6b7 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:02:26.399366 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1294/2568] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 04:02:26 2022] POST /identity/v3/users => generated 290 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 04:02:26.403531 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbcb5df-edc0-47c4-8b7b-83c685353c5a admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:02:26.414267 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbcb5df-edc0-47c4-8b7b-83c685353c5a admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:02:26.418943 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d14399-0f43-4e5a-9b4a-19068e7efce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:02:26.419536 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d14399-0f43-4e5a-9b4a-19068e7efce4 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:26.419854 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d14399-0f43-4e5a-9b4a-19068e7efce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:26.420128 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d14399-0f43-4e5a-9b4a-19068e7efce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:26.420898 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d14399-0f43-4e5a-9b4a-19068e7efce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:02:26.423363 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d14399-0f43-4e5a-9b4a-19068e7efce4 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:02:26.433119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1275/2569] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 04:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 04:02:26.436951 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf5ed6-ffde-4c53-859b-fec92959d6b7 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:02:26.446776 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf5ed6-ffde-4c53-859b-fec92959d6b7 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:02:26.451453 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d8faf-ab1d-41dd-8e37-84b22d8ce0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:02:26.452031 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d8faf-ab1d-41dd-8e37-84b22d8ce0d7 admin admin] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:26.452311 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d8faf-ab1d-41dd-8e37-84b22d8ce0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:26.452583 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d8faf-ab1d-41dd-8e37-84b22d8ce0d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:26.453226 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d8faf-ab1d-41dd-8e37-84b22d8ce0d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:02:26.454793 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3d8faf-ab1d-41dd-8e37-84b22d8ce0d7 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:02:26.465341 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1295/2570] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 04:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 04:02:26.470072 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d14399-0f43-4e5a-9b4a-19068e7efce4 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:02:26.479894 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d14399-0f43-4e5a-9b4a-19068e7efce4 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:02:26.485021 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71231b-8c28-48de-8c6f-114559d0e47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:02:26.485663 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71231b-8c28-48de-8c6f-114559d0e47d admin admin] REQUEST_METHOD: `PUT` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:26.485945 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71231b-8c28-48de-8c6f-114559d0e47d admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:26.486223 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d71231b-8c28-48de-8c6f-114559d0e47d admin admin] PATH_INFO: `/v3/projects/8dbea8f697124e6bb449a9ed71670dda/users/8b0bb095424a4e8abc828e0b529d9bc5/roles/84a147decc7b417783086c1effe0fe20` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:26.501554 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d71231b-8c28-48de-8c6f-114559d0e47d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dbea8f697124e6bb449a9ed71670dda, user_id=8b0bb095424a4e8abc828e0b529d9bc5, role_id=84a147decc7b417783086c1effe0fe20)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:02:26.504187 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d71231b-8c28-48de-8c6f-114559d0e47d admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:02:26.611780 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1276/2571] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 04:02:26 2022] PUT /identity/v3/projects/8dbea8f697124e6bb449a9ed71670dda/users/8b0bb095424a4e8abc828e0b529d9bc5/roles/84a147decc7b417783086c1effe0fe20 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 04:02:26.617510 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeabaac-81ba-4a80-909c-ee31b6e749b3 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:26.617801 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeabaac-81ba-4a80-909c-ee31b6e749b3 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:26.618552 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeabaac-81ba-4a80-909c-ee31b6e749b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:26.637833 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-abeabaac-81ba-4a80-909c-ee31b6e749b3 None None] MFA Rules not processed for user `f09f98306cd54b068020428d74bd0957`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 04:02:26.639144 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-abeabaac-81ba-4a80-909c-ee31b6e749b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:02:26.698587 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1296/2572] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 04:02:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 04:02:26.708278 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71231b-8c28-48de-8c6f-114559d0e47d admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:02:26.709917 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-5d71231b-8c28-48de-8c6f-114559d0e47d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:02:26.767119 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d71231b-8c28-48de-8c6f-114559d0e47d admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:02:26.770776 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5d2df-a2da-4c86-8ab7-73d12a571805 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:02:26.771376 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5d2df-a2da-4c86-8ab7-73d12a571805 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:26.771662 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5d2df-a2da-4c86-8ab7-73d12a571805 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:26.771965 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5d2df-a2da-4c86-8ab7-73d12a571805 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:26.781386 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc5d2df-a2da-4c86-8ab7-73d12a571805 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:02:26.782794 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc5d2df-a2da-4c86-8ab7-73d12a571805 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:02:26.792013 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1277/2573] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:02:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:02:31.918829 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a880e9-210d-4fdf-b797-ec776ef51886 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:31.919445 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a880e9-210d-4fdf-b797-ec776ef51886 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:31.919445 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a880e9-210d-4fdf-b797-ec776ef51886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:31.952132 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-05a880e9-210d-4fdf-b797-ec776ef51886 None None] MFA Rules not processed for user `8b0bb095424a4e8abc828e0b529d9bc5`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 04:02:31.953371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-05a880e9-210d-4fdf-b797-ec776ef51886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:02:32.005565 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:02:32.023419 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1297/2574] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 04:02:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 04:02:32.028001 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5d2df-a2da-4c86-8ab7-73d12a571805 service neutron] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:02:32.038142 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5d2df-a2da-4c86-8ab7-73d12a571805 service neutron] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:02:32.043194 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990938bf-a2a7-4d6a-abe7-2e2f026162f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:02:32.043824 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990938bf-a2a7-4d6a-abe7-2e2f026162f0 admin admin] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:32.044109 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990938bf-a2a7-4d6a-abe7-2e2f026162f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:32.044386 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990938bf-a2a7-4d6a-abe7-2e2f026162f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:32.045034 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990938bf-a2a7-4d6a-abe7-2e2f026162f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:02:32.047483 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990938bf-a2a7-4d6a-abe7-2e2f026162f0 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:02:32.126600 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1278/2575] 10.222.0.17 () {60 vars in 1153 bytes} [Mon Oct 17 04:02:32 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 04:02:32.130508 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a880e9-210d-4fdf-b797-ec776ef51886 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:02:32.142007 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a880e9-210d-4fdf-b797-ec776ef51886 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:02:32.207413 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052148a-3ad9-4a93-a8ad-de2493590caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:02:32.207413 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052148a-3ad9-4a93-a8ad-de2493590caf admin admin] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:32.207413 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052148a-3ad9-4a93-a8ad-de2493590caf admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:32.207413 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052148a-3ad9-4a93-a8ad-de2493590caf admin admin] PATH_INFO: `/v3/users` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:32.208115 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e052148a-3ad9-4a93-a8ad-de2493590caf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:02:32.208115 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e052148a-3ad9-4a93-a8ad-de2493590caf admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:02:32.270367 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1298/2576] 10.222.0.17 () {60 vars in 1144 bytes} [Mon Oct 17 04:02:32 2022] POST /identity/v3/users => generated 296 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 17 04:02:32.271582 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990938bf-a2a7-4d6a-abe7-2e2f026162f0 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:02:32.283287 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990938bf-a2a7-4d6a-abe7-2e2f026162f0 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:02:32.288237 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c95bb8-7ab9-45e0-936e-4dee2ca2f276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:02:32.288863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c95bb8-7ab9-45e0-936e-4dee2ca2f276 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:32.289147 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c95bb8-7ab9-45e0-936e-4dee2ca2f276 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:32.289499 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c95bb8-7ab9-45e0-936e-4dee2ca2f276 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:32.290164 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c95bb8-7ab9-45e0-936e-4dee2ca2f276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:02:32.291781 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c95bb8-7ab9-45e0-936e-4dee2ca2f276 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:02:32.303694 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1279/2577] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 04:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 04:02:32.308003 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052148a-3ad9-4a93-a8ad-de2493590caf admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:02:32.318032 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052148a-3ad9-4a93-a8ad-de2493590caf admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:02:32.322594 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061f9a8-269b-4f8c-be14-35357a7fea60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:02:32.323237 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061f9a8-269b-4f8c-be14-35357a7fea60 admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:32.323512 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061f9a8-269b-4f8c-be14-35357a7fea60 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:32.323805 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061f9a8-269b-4f8c-be14-35357a7fea60 admin admin] PATH_INFO: `/v3/projects/a48f898280ab4371a905047720968afa/users/099351bc2fbe4f3bb6dc51aa4cf13b13/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:32.459258 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061f9a8-269b-4f8c-be14-35357a7fea60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a48f898280ab4371a905047720968afa, user_id=099351bc2fbe4f3bb6dc51aa4cf13b13, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:02:32.461966 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061f9a8-269b-4f8c-be14-35357a7fea60 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:02:32.531972 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1299/2578] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 04:02:32 2022] PUT /identity/v3/projects/a48f898280ab4371a905047720968afa/users/099351bc2fbe4f3bb6dc51aa4cf13b13/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 04:02:32.535985 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c95bb8-7ab9-45e0-936e-4dee2ca2f276 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:02:32.546636 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c95bb8-7ab9-45e0-936e-4dee2ca2f276 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:02:32.602110 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a58bdc5-64de-4b18-a8d5-dd61786948f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:02:32.602762 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58bdc5-64de-4b18-a8d5-dd61786948f5 admin admin] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:32.603028 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58bdc5-64de-4b18-a8d5-dd61786948f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:32.603296 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58bdc5-64de-4b18-a8d5-dd61786948f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:32.603959 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a58bdc5-64de-4b18-a8d5-dd61786948f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:02:32.605522 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a58bdc5-64de-4b18-a8d5-dd61786948f5 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:02:32.615607 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1280/2579] 10.222.0.17 () {58 vars in 1122 bytes} [Mon Oct 17 04:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 17 04:02:32.619657 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061f9a8-269b-4f8c-be14-35357a7fea60 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:02:32.629798 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061f9a8-269b-4f8c-be14-35357a7fea60 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:02:32.634325 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f8f68-dda9-4615-8045-1f8fc030991f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:02:32.634962 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f8f68-dda9-4615-8045-1f8fc030991f admin admin] REQUEST_METHOD: `PUT` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:32.635246 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f8f68-dda9-4615-8045-1f8fc030991f admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:32.635515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086f8f68-dda9-4615-8045-1f8fc030991f admin admin] PATH_INFO: `/v3/projects/a48f898280ab4371a905047720968afa/users/099351bc2fbe4f3bb6dc51aa4cf13b13/roles/611120afb3a544d0b5161da1b769a2f8` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:32.637417 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f8f68-dda9-4615-8045-1f8fc030991f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a48f898280ab4371a905047720968afa, user_id=099351bc2fbe4f3bb6dc51aa4cf13b13, role_id=611120afb3a544d0b5161da1b769a2f8)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:02:32.640548 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086f8f68-dda9-4615-8045-1f8fc030991f admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:02:32.650577 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1300/2580] 10.222.0.17 () {60 vars in 1483 bytes} [Mon Oct 17 04:02:32 2022] PUT /identity/v3/projects/a48f898280ab4371a905047720968afa/users/099351bc2fbe4f3bb6dc51aa4cf13b13/roles/611120afb3a544d0b5161da1b769a2f8 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 04:02:37.959704 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a403a6-9cc9-4959-9fe7-e0ad1df14ae7 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:37.960295 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a403a6-9cc9-4959-9fe7-e0ad1df14ae7 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:37.960295 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a403a6-9cc9-4959-9fe7-e0ad1df14ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:37.991636 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-19a403a6-9cc9-4959-9fe7-e0ad1df14ae7 None None] MFA Rules not processed for user `099351bc2fbe4f3bb6dc51aa4cf13b13`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 04:02:37.993012 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-19a403a6-9cc9-4959-9fe7-e0ad1df14ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:02:37.997162 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:02:38.072854 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1281/2581] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 04:02:37 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 04:02:38.089515 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f8f68-dda9-4615-8045-1f8fc030991f admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:02:38.091582 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-086f8f68-dda9-4615-8045-1f8fc030991f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:02:38.172429 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086f8f68-dda9-4615-8045-1f8fc030991f admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:02:38.177993 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c30fb-ef3e-43e0-8350-66d5f18b61f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:02:38.178613 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c30fb-ef3e-43e0-8350-66d5f18b61f8 service nova] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:38.178891 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c30fb-ef3e-43e0-8350-66d5f18b61f8 service nova] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:38.179161 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c30fb-ef3e-43e0-8350-66d5f18b61f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:38.223585 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:02:38.239716 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97c30fb-ef3e-43e0-8350-66d5f18b61f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:02:38.241335 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97c30fb-ef3e-43e0-8350-66d5f18b61f8 service nova] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:02:38.251318 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1301/2582] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:02:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:02:48.106618 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=27 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:02:48.284831 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=9 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:02:57.241893 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27171c0b-a0d5-4cac-a838-7b897b620a0f None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:57.242401 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27171c0b-a0d5-4cac-a838-7b897b620a0f None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:57.242505 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27171c0b-a0d5-4cac-a838-7b897b620a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:57.260950 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-27171c0b-a0d5-4cac-a838-7b897b620a0f None None] MFA Rules not processed for user `8b0bb095424a4e8abc828e0b529d9bc5`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 04:02:57.262123 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-27171c0b-a0d5-4cac-a838-7b897b620a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:02:57.268776 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1282/2583] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 04:02:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 04:02:57.572660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c30fb-ef3e-43e0-8350-66d5f18b61f8 service nova] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:02:57.629333 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c30fb-ef3e-43e0-8350-66d5f18b61f8 service nova] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:02:57.632897 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322191b-4df3-480b-a4f0-fd2f37e4b42f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:02:57.633506 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322191b-4df3-480b-a4f0-fd2f37e4b42f service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:57.633783 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322191b-4df3-480b-a4f0-fd2f37e4b42f service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:57.634050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322191b-4df3-480b-a4f0-fd2f37e4b42f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:57.635877 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-4322191b-4df3-480b-a4f0-fd2f37e4b42f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:02:57.645773 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322191b-4df3-480b-a4f0-fd2f37e4b42f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:02:57.647194 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322191b-4df3-480b-a4f0-fd2f37e4b42f service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:02:57.655567 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1302/2584] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:02:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:02:58.349841 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27171c0b-a0d5-4cac-a838-7b897b620a0f None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:02:58.351263 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-27171c0b-a0d5-4cac-a838-7b897b620a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:02:58.424475 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27171c0b-a0d5-4cac-a838-7b897b620a0f None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:02:58.429357 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445e935-c7b6-4c4b-9cac-be9c468818eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b500a113a2aa4a95875d7e094b917db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:02:58.429967 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445e935-c7b6-4c4b-9cac-be9c468818eb service placement] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:02:58.430280 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445e935-c7b6-4c4b-9cac-be9c468818eb service placement] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:02:58.430581 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1445e935-c7b6-4c4b-9cac-be9c468818eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:02:58.432425 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-1445e935-c7b6-4c4b-9cac-be9c468818eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:02:58.444594 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1445e935-c7b6-4c4b-9cac-be9c468818eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:02:58.446207 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1445e935-c7b6-4c4b-9cac-be9c468818eb service placement] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:02:58.456127 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1283/2585] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:02:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:03:02.380394 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322191b-4df3-480b-a4f0-fd2f37e4b42f service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:03:02.381908 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-4322191b-4df3-480b-a4f0-fd2f37e4b42f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:03:02.392086 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322191b-4df3-480b-a4f0-fd2f37e4b42f service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:03:02.396050 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6532f1e2-7997-4eff-9614-cda66473ea30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:03:02.396759 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532f1e2-7997-4eff-9614-cda66473ea30 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:03:02.397090 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532f1e2-7997-4eff-9614-cda66473ea30 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:03:02.397426 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532f1e2-7997-4eff-9614-cda66473ea30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:03:02.399224 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6532f1e2-7997-4eff-9614-cda66473ea30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:03:02.410199 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6532f1e2-7997-4eff-9614-cda66473ea30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:03:02.411719 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6532f1e2-7997-4eff-9614-cda66473ea30 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:03:02.420882 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1303/2586] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:03:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:03:02.550052 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445e935-c7b6-4c4b-9cac-be9c468818eb service placement] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:03:02.560021 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1445e935-c7b6-4c4b-9cac-be9c468818eb service placement] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:03:02.563668 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb878f74-1048-4d25-b969-1d6aefbe68b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:03:02.564304 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb878f74-1048-4d25-b969-1d6aefbe68b7 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:03:02.564587 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb878f74-1048-4d25-b969-1d6aefbe68b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:03:02.564863 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb878f74-1048-4d25-b969-1d6aefbe68b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:03:02.571642 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-eb878f74-1048-4d25-b969-1d6aefbe68b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:03:02.587360 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb878f74-1048-4d25-b969-1d6aefbe68b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:03:02.588833 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb878f74-1048-4d25-b969-1d6aefbe68b7 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:03:02.608263 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1284/2587] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:03:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:03:03.778841 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6532f1e2-7997-4eff-9614-cda66473ea30 service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:03:03.780649 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-6532f1e2-7997-4eff-9614-cda66473ea30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:03:03.797576 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6532f1e2-7997-4eff-9614-cda66473ea30 service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:03:03.804193 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db72b4-5750-4080-8601-a100810aac09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:03:03.804855 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db72b4-5750-4080-8601-a100810aac09 service nova] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:03:03.805128 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db72b4-5750-4080-8601-a100810aac09 service nova] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:03:03.805489 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db72b4-5750-4080-8601-a100810aac09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:03:03.807209 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-21db72b4-5750-4080-8601-a100810aac09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:03:03.819803 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21db72b4-5750-4080-8601-a100810aac09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:03:03.821522 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21db72b4-5750-4080-8601-a100810aac09 service nova] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:03:03.833231 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1304/2588] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:03:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:03:12.596250 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=41 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:03:13.827910 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=38 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:03:18.564043 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb878f74-1048-4d25-b969-1d6aefbe68b7 service neutron] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:03:18.573210 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb878f74-1048-4d25-b969-1d6aefbe68b7 service neutron] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:03:18.576869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048ba62-9d36-45ce-b6fb-467138690572 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:03:18.577461 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048ba62-9d36-45ce-b6fb-467138690572 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:03:18.577745 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048ba62-9d36-45ce-b6fb-467138690572 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:03:18.578016 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048ba62-9d36-45ce-b6fb-467138690572 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:03:18.586676 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048ba62-9d36-45ce-b6fb-467138690572 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:03:18.588117 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048ba62-9d36-45ce-b6fb-467138690572 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:03:18.597578 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1285/2589] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:03:18 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:03:28.593454 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=6 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:04:00.845952 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db72b4-5750-4080-8601-a100810aac09 service nova] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:04:00.847328 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-21db72b4-5750-4080-8601-a100810aac09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:04:00.861371 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db72b4-5750-4080-8601-a100810aac09 service nova] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:04:00.866789 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5cfd9-0567-43e1-99f5-5560149488bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b500a113a2aa4a95875d7e094b917db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:04:00.867378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5cfd9-0567-43e1-99f5-5560149488bb service placement] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:04:00.867660 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5cfd9-0567-43e1-99f5-5560149488bb service placement] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:04:00.867964 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5cfd9-0567-43e1-99f5-5560149488bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:04:00.869628 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-bdf5cfd9-0567-43e1-99f5-5560149488bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:04:00.881006 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5cfd9-0567-43e1-99f5-5560149488bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:04:00.883226 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5cfd9-0567-43e1-99f5-5560149488bb service placement] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:04:00.893075 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1305/2590] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:04:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:04:10.888884 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=6 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:04:48.368160 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048ba62-9d36-45ce-b6fb-467138690572 service neutron] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:04:48.379241 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048ba62-9d36-45ce-b6fb-467138690572 service neutron] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:04:48.384734 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f5682-1f0b-4ce6-a8a4-b5aeba82a91e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b500a113a2aa4a95875d7e094b917db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:04:48.385334 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f5682-1f0b-4ce6-a8a4-b5aeba82a91e service placement] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:04:48.385617 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f5682-1f0b-4ce6-a8a4-b5aeba82a91e service placement] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:04:48.385893 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427f5682-1f0b-4ce6-a8a4-b5aeba82a91e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:04:48.387598 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-427f5682-1f0b-4ce6-a8a4-b5aeba82a91e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:04:48.399239 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427f5682-1f0b-4ce6-a8a4-b5aeba82a91e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:04:48.400868 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427f5682-1f0b-4ce6-a8a4-b5aeba82a91e service placement] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:04:48.411204 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1286/2591] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:04:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:04:51.767088 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5cfd9-0567-43e1-99f5-5560149488bb service placement] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:04:51.768614 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-bdf5cfd9-0567-43e1-99f5-5560149488bb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:04:51.833577 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5cfd9-0567-43e1-99f5-5560149488bb service placement] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:04:51.838126 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f619f-cbab-41b4-868f-ed0e5475a2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:04:51.838752 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f619f-cbab-41b4-868f-ed0e5475a2b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:04:51.839031 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f619f-cbab-41b4-868f-ed0e5475a2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:04:51.839334 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f619f-cbab-41b4-868f-ed0e5475a2b8 admin admin] PATH_INFO: `/v3/users/926a7f570adb46debe57caefead0e17b` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:04:51.840500 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f619f-cbab-41b4-868f-ed0e5475a2b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=926a7f570adb46debe57caefead0e17b)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:04:51.842651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f619f-cbab-41b4-868f-ed0e5475a2b8 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:04:51.969723 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-931f619f-cbab-41b4-868f-ed0e5475a2b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '926a7f570adb46debe57caefead0e17b'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:04:51.970370 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-931f619f-cbab-41b4-868f-ed0e5475a2b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '926a7f570adb46debe57caefead0e17b'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:04:52.012855 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-931f619f-cbab-41b4-868f-ed0e5475a2b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '926a7f570adb46debe57caefead0e17b'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:04:52.025221 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-931f619f-cbab-41b4-868f-ed0e5475a2b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '926a7f570adb46debe57caefead0e17b'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:04:52.025854 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-931f619f-cbab-41b4-868f-ed0e5475a2b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '926a7f570adb46debe57caefead0e17b'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:04:52.038715 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1306/2592] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 04:04:51 2022] DELETE /identity/v3/users/926a7f570adb46debe57caefead0e17b => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 04:04:52.048346 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f5682-1f0b-4ce6-a8a4-b5aeba82a91e service placement] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:04:52.057770 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427f5682-1f0b-4ce6-a8a4-b5aeba82a91e service placement] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:04:52.105655 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0e74b-e4ce-44ce-950b-4df5a7e7b35b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:04:52.106247 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0e74b-e4ce-44ce-950b-4df5a7e7b35b service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:04:52.106522 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0e74b-e4ce-44ce-950b-4df5a7e7b35b service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:04:52.106795 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0e74b-e4ce-44ce-950b-4df5a7e7b35b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:04:52.108526 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-bde0e74b-e4ce-44ce-950b-4df5a7e7b35b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:04:52.168796 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0e74b-e4ce-44ce-950b-4df5a7e7b35b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:04:52.170208 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0e74b-e4ce-44ce-950b-4df5a7e7b35b service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:04:52.180016 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1287/2593] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:04:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:04:52.414654 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f619f-cbab-41b4-868f-ed0e5475a2b8 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:04:52.416077 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-931f619f-cbab-41b4-868f-ed0e5475a2b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:04:52.429237 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f619f-cbab-41b4-868f-ed0e5475a2b8 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:04:52.434122 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a646fe-246b-4bd1-9049-618020d0c517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:04:52.434758 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a646fe-246b-4bd1-9049-618020d0c517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:04:52.435039 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a646fe-246b-4bd1-9049-618020d0c517 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:04:52.435302 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a646fe-246b-4bd1-9049-618020d0c517 admin admin] PATH_INFO: `/v3/projects/a64255e69ce9468e9a7bef2297b17725` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:04:52.436433 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a646fe-246b-4bd1-9049-618020d0c517 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a64255e69ce9468e9a7bef2297b17725)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:04:52.438306 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a646fe-246b-4bd1-9049-618020d0c517 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:04:52.521587 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-54a646fe-246b-4bd1-9049-618020d0c517 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a64255e69ce9468e9a7bef2297b17725', 'request_id': 'req-54a646fe-246b-4bd1-9049-618020d0c517'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:04:52.572916 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-54a646fe-246b-4bd1-9049-618020d0c517 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a64255e69ce9468e9a7bef2297b17725', 'request_id': 'req-54a646fe-246b-4bd1-9049-618020d0c517'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:04:52.578471 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-54a646fe-246b-4bd1-9049-618020d0c517 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a64255e69ce9468e9a7bef2297b17725', 'request_id': 'req-54a646fe-246b-4bd1-9049-618020d0c517'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:04:52.590039 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-54a646fe-246b-4bd1-9049-618020d0c517 admin admin] The token cache is being invalidate because project a64255e69ce9468e9a7bef2297b17725 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 04:04:52.590322 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-54a646fe-246b-4bd1-9049-618020d0c517 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:04:52.590936 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-54a646fe-246b-4bd1-9049-618020d0c517 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:04:52.592737 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1307/2594] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 04:04:52 2022] DELETE /identity/v3/projects/a64255e69ce9468e9a7bef2297b17725 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 04:04:52.596707 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0e74b-e4ce-44ce-950b-4df5a7e7b35b service neutron] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:04:52.606280 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0e74b-e4ce-44ce-950b-4df5a7e7b35b service neutron] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:04:52.661317 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443fde4a-548f-4464-a2e2-99b83fcddd1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:04:52.661937 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443fde4a-548f-4464-a2e2-99b83fcddd1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:04:52.662216 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443fde4a-548f-4464-a2e2-99b83fcddd1b admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:04:52.662490 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443fde4a-548f-4464-a2e2-99b83fcddd1b admin admin] PATH_INFO: `/v3/users/de56df051ac546dcb3f5c1d43e348f7e` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:04:52.663592 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443fde4a-548f-4464-a2e2-99b83fcddd1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=de56df051ac546dcb3f5c1d43e348f7e)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:04:52.665524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443fde4a-548f-4464-a2e2-99b83fcddd1b admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:04:52.826778 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:04:52.832476 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-443fde4a-548f-4464-a2e2-99b83fcddd1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de56df051ac546dcb3f5c1d43e348f7e'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:04:52.904072 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-443fde4a-548f-4464-a2e2-99b83fcddd1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de56df051ac546dcb3f5c1d43e348f7e'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:04:52.916104 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-443fde4a-548f-4464-a2e2-99b83fcddd1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de56df051ac546dcb3f5c1d43e348f7e'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:04:52.916775 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-443fde4a-548f-4464-a2e2-99b83fcddd1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de56df051ac546dcb3f5c1d43e348f7e'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:04:52.917382 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-443fde4a-548f-4464-a2e2-99b83fcddd1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de56df051ac546dcb3f5c1d43e348f7e'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:04:52.930116 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1288/2595] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 04:04:52 2022] DELETE /identity/v3/users/de56df051ac546dcb3f5c1d43e348f7e => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 04:04:53.180919 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a646fe-246b-4bd1-9049-618020d0c517 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:04:53.182844 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-54a646fe-246b-4bd1-9049-618020d0c517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:04:53.191626 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:04:53.197297 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a646fe-246b-4bd1-9049-618020d0c517 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:04:53.253741 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78228aa-4859-42c8-b92f-7d29f86187ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:04:53.254373 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78228aa-4859-42c8-b92f-7d29f86187ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:04:53.254651 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78228aa-4859-42c8-b92f-7d29f86187ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:04:53.254922 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78228aa-4859-42c8-b92f-7d29f86187ba admin admin] PATH_INFO: `/v3/projects/2f05de57855e405cb4ff423353bc85c7` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:04:53.256072 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78228aa-4859-42c8-b92f-7d29f86187ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f05de57855e405cb4ff423353bc85c7)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:04:53.257960 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78228aa-4859-42c8-b92f-7d29f86187ba admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:04:53.339379 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-c78228aa-4859-42c8-b92f-7d29f86187ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f05de57855e405cb4ff423353bc85c7', 'request_id': 'req-c78228aa-4859-42c8-b92f-7d29f86187ba'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:04:53.401360 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-c78228aa-4859-42c8-b92f-7d29f86187ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f05de57855e405cb4ff423353bc85c7', 'request_id': 'req-c78228aa-4859-42c8-b92f-7d29f86187ba'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:04:53.407004 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-c78228aa-4859-42c8-b92f-7d29f86187ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f05de57855e405cb4ff423353bc85c7', 'request_id': 'req-c78228aa-4859-42c8-b92f-7d29f86187ba'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:04:53.418526 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-c78228aa-4859-42c8-b92f-7d29f86187ba admin admin] The token cache is being invalidate because project 2f05de57855e405cb4ff423353bc85c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88299) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 04:04:53.418746 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-c78228aa-4859-42c8-b92f-7d29f86187ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:04:53.419368 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-c78228aa-4859-42c8-b92f-7d29f86187ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:04:53.420852 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1308/2596] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 04:04:53 2022] DELETE /identity/v3/projects/2f05de57855e405cb4ff423353bc85c7 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 04:05:02.996485 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:DELETE=5,keystone:SELECT=14,keystone:INSERT=1 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:05:03.465461 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=50,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:05:22.931504 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2460af4-2b09-4c4b-9504-9cec6671e735 None None] REQUEST_METHOD: `POST` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:05:22.932066 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2460af4-2b09-4c4b-9504-9cec6671e735 None None] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:05:22.932066 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2460af4-2b09-4c4b-9504-9cec6671e735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:05:22.952020 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.auth.core [None req-a2460af4-2b09-4c4b-9504-9cec6671e735 None None] MFA Rules not processed for user `8b0bb095424a4e8abc828e0b529d9bc5`. Rule list: `[]` (Enabled: `True`). {{(pid=88300) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 04:05:22.953169 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-a2460af4-2b09-4c4b-9504-9cec6671e735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:05:23.007441 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1289/2597] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 04:05:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 04:05:23.314807 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78228aa-4859-42c8-b92f-7d29f86187ba admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:05:23.316228 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-c78228aa-4859-42c8-b92f-7d29f86187ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:05:23.372158 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78228aa-4859-42c8-b92f-7d29f86187ba admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:05:23.375612 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980043d-8e6b-4434-ba00-49e95f16e876 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:05:23.376229 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980043d-8e6b-4434-ba00-49e95f16e876 service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:05:23.376506 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980043d-8e6b-4434-ba00-49e95f16e876 service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:05:23.376772 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c980043d-8e6b-4434-ba00-49e95f16e876 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:05:23.378435 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-c980043d-8e6b-4434-ba00-49e95f16e876 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:05:23.387666 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c980043d-8e6b-4434-ba00-49e95f16e876 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:05:23.397150 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c980043d-8e6b-4434-ba00-49e95f16e876 service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:05:23.397536 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1309/2598] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:05:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:05:33.000637 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=25 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:05:33.399307 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=26 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:05:43.493729 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2460af4-2b09-4c4b-9504-9cec6671e735 None None] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:05:43.495121 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-a2460af4-2b09-4c4b-9504-9cec6671e735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:05:43.507509 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2460af4-2b09-4c4b-9504-9cec6671e735 None None] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:05:43.511045 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b78a3-56ea-419f-a4bf-1a69cc51b68f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:05:43.511647 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b78a3-56ea-419f-a4bf-1a69cc51b68f service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:05:43.511962 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b78a3-56ea-419f-a4bf-1a69cc51b68f service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:05:43.512238 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329b78a3-56ea-419f-a4bf-1a69cc51b68f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:05:43.513885 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-329b78a3-56ea-419f-a4bf-1a69cc51b68f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:05:43.583102 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329b78a3-56ea-419f-a4bf-1a69cc51b68f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:05:43.584545 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329b78a3-56ea-419f-a4bf-1a69cc51b68f service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:05:43.595788 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1290/2599] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:05:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:05:43.773378 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980043d-8e6b-4434-ba00-49e95f16e876 service neutron] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:05:43.781938 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c980043d-8e6b-4434-ba00-49e95f16e876 service neutron] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:05:43.785508 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5232949f-dd80-4f89-a29f-836998b0edbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:05:43.786094 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232949f-dd80-4f89-a29f-836998b0edbb service neutron] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:05:43.786372 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232949f-dd80-4f89-a29f-836998b0edbb service neutron] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:05:43.786641 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5232949f-dd80-4f89-a29f-836998b0edbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:05:43.788339 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-5232949f-dd80-4f89-a29f-836998b0edbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:05:43.801755 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5232949f-dd80-4f89-a29f-836998b0edbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:05:43.803153 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5232949f-dd80-4f89-a29f-836998b0edbb service neutron] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:05:43.813790 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1310/2600] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:05:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:05:53.590543 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=34 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:05:53.809014 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=6 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:07:04.249327 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b78a3-56ea-419f-a4bf-1a69cc51b68f service neutron] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:07:04.258622 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329b78a3-56ea-419f-a4bf-1a69cc51b68f service neutron] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:07:04.262265 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f089a5b-4c2b-4573-8c7a-76df68c3bb4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:07:04.262865 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f089a5b-4c2b-4573-8c7a-76df68c3bb4e service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:07:04.263147 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f089a5b-4c2b-4573-8c7a-76df68c3bb4e service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:07:04.263424 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f089a5b-4c2b-4573-8c7a-76df68c3bb4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:07:04.265213 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-2f089a5b-4c2b-4573-8c7a-76df68c3bb4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:07:04.275202 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f089a5b-4c2b-4573-8c7a-76df68c3bb4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:07:04.276647 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f089a5b-4c2b-4573-8c7a-76df68c3bb4e service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:07:04.284480 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1291/2601] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:07:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:07:14.280884 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=6 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:07:24.565148 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0afee-87df-471f-873a-ac6863e980f3 None None] REQUEST_METHOD: `POST` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:07:24.565687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0afee-87df-471f-873a-ac6863e980f3 None None] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:07:24.565687 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0afee-87df-471f-873a-ac6863e980f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:07:24.585390 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.auth.core [None req-77e0afee-87df-471f-873a-ac6863e980f3 None None] MFA Rules not processed for user `8b0bb095424a4e8abc828e0b529d9bc5`. Rule list: `[]` (Enabled: `True`). {{(pid=88299) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 17 04:07:24.586567 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-77e0afee-87df-471f-873a-ac6863e980f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:07:24.594162 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1311/2602] 10.222.0.17 () {58 vars in 958 bytes} [Mon Oct 17 04:07:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 17 04:07:29.322314 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f089a5b-4c2b-4573-8c7a-76df68c3bb4e service neutron] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:07:29.323767 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-2f089a5b-4c2b-4573-8c7a-76df68c3bb4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:07:29.339869 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f089a5b-4c2b-4573-8c7a-76df68c3bb4e service neutron] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:07:29.345702 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd523d-ecc1-4ad0-8eef-f53aeea9d216 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:07:29.346317 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd523d-ecc1-4ad0-8eef-f53aeea9d216 service nova] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:07:29.346599 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd523d-ecc1-4ad0-8eef-f53aeea9d216 service nova] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:07:29.346883 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cd523d-ecc1-4ad0-8eef-f53aeea9d216 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:07:29.348681 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-01cd523d-ecc1-4ad0-8eef-f53aeea9d216 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:07:29.361293 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd523d-ecc1-4ad0-8eef-f53aeea9d216 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:07:29.362897 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cd523d-ecc1-4ad0-8eef-f53aeea9d216 service nova] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:07:29.375876 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1292/2603] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:07:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:07:34.580020 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=3 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:07:38.741313 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0afee-87df-471f-873a-ac6863e980f3 None None] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:07:38.753856 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0afee-87df-471f-873a-ac6863e980f3 None None] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:07:38.759125 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb0200-5ad3-4024-bcef-4cedd7ea6f7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:07:38.759711 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb0200-5ad3-4024-bcef-4cedd7ea6f7c service nova] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:07:38.760016 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb0200-5ad3-4024-bcef-4cedd7ea6f7c service nova] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:07:38.760283 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb0200-5ad3-4024-bcef-4cedd7ea6f7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:07:38.761963 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-aebb0200-5ad3-4024-bcef-4cedd7ea6f7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:07:38.772125 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb0200-5ad3-4024-bcef-4cedd7ea6f7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:07:38.773665 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb0200-5ad3-4024-bcef-4cedd7ea6f7c service nova] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:07:38.782149 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1312/2604] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:07:38 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:07:39.370947 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=6 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:07:43.517697 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd523d-ecc1-4ad0-8eef-f53aeea9d216 service nova] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:07:43.519178 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-01cd523d-ecc1-4ad0-8eef-f53aeea9d216 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:07:43.532851 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cd523d-ecc1-4ad0-8eef-f53aeea9d216 service nova] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:07:43.538402 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d91469-b02c-4816-b16f-7d25f1ea548a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd961b751763a4e49b5d4ca4c795c5bf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:07:43.539014 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d91469-b02c-4816-b16f-7d25f1ea548a service nova] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:07:43.539291 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d91469-b02c-4816-b16f-7d25f1ea548a service nova] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:07:43.539560 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d91469-b02c-4816-b16f-7d25f1ea548a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:07:43.541302 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-62d91469-b02c-4816-b16f-7d25f1ea548a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:07:43.554773 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d91469-b02c-4816-b16f-7d25f1ea548a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:07:43.556397 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d91469-b02c-4816-b16f-7d25f1ea548a service nova] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:07:43.568891 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1293/2605] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:07:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:07:48.778348 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=6 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:07:53.563224 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=6 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:09:06.451403 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb0200-5ad3-4024-bcef-4cedd7ea6f7c service nova] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:09:06.452825 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-aebb0200-5ad3-4024-bcef-4cedd7ea6f7c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:09:06.585481 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb0200-5ad3-4024-bcef-4cedd7ea6f7c service nova] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:09:06.591185 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc5c94-10be-440c-8f3a-2a38c6214245 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b500a113a2aa4a95875d7e094b917db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:09:06.591802 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc5c94-10be-440c-8f3a-2a38c6214245 service placement] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:09:06.592082 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc5c94-10be-440c-8f3a-2a38c6214245 service placement] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:09:06.592357 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc5c94-10be-440c-8f3a-2a38c6214245 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:09:06.594054 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-56cc5c94-10be-440c-8f3a-2a38c6214245 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:09:06.605719 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cc5c94-10be-440c-8f3a-2a38c6214245 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:09:06.607253 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cc5c94-10be-440c-8f3a-2a38c6214245 service placement] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:09:06.617176 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1313/2606] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:09:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:09:16.612735 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=63 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:09:37.173373 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d91469-b02c-4816-b16f-7d25f1ea548a service nova] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:09:37.209549 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d91469-b02c-4816-b16f-7d25f1ea548a service nova] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:09:37.213267 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd00e64-8c10-4238-9d37-aa6668208291 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '414707c545f641d3935720356b326672', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:09:37.213912 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd00e64-8c10-4238-9d37-aa6668208291 service neutron] REQUEST_METHOD: `GET` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:09:37.214190 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd00e64-8c10-4238-9d37-aa6668208291 service neutron] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:09:37.214459 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd00e64-8c10-4238-9d37-aa6668208291 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:09:37.216944 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-bfd00e64-8c10-4238-9d37-aa6668208291 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:09:37.299482 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd00e64-8c10-4238-9d37-aa6668208291 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:09:37.300920 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd00e64-8c10-4238-9d37-aa6668208291 service neutron] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:09:37.318992 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1294/2607] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:09:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:09:44.577264 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc5c94-10be-440c-8f3a-2a38c6214245 service placement] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:09:44.578750 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-56cc5c94-10be-440c-8f3a-2a38c6214245 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:09:44.592786 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc5c94-10be-440c-8f3a-2a38c6214245 service placement] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:09:44.597494 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a0c11-44d4-4889-9a6b-76dd9e7eeda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:09:44.598525 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a0c11-44d4-4889-9a6b-76dd9e7eeda3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:09:44.598525 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a0c11-44d4-4889-9a6b-76dd9e7eeda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:09:44.598754 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a0c11-44d4-4889-9a6b-76dd9e7eeda3 admin admin] PATH_INFO: `/v3/users/8b0bb095424a4e8abc828e0b529d9bc5` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:09:44.599765 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879a0c11-44d4-4889-9a6b-76dd9e7eeda3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b0bb095424a4e8abc828e0b529d9bc5)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:09:44.601671 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879a0c11-44d4-4889-9a6b-76dd9e7eeda3 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:09:44.800526 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-879a0c11-44d4-4889-9a6b-76dd9e7eeda3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b0bb095424a4e8abc828e0b529d9bc5'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:09:44.801178 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-879a0c11-44d4-4889-9a6b-76dd9e7eeda3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b0bb095424a4e8abc828e0b529d9bc5'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:09:44.885145 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-879a0c11-44d4-4889-9a6b-76dd9e7eeda3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b0bb095424a4e8abc828e0b529d9bc5'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:09:44.897942 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-879a0c11-44d4-4889-9a6b-76dd9e7eeda3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b0bb095424a4e8abc828e0b529d9bc5'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:09:44.898571 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-879a0c11-44d4-4889-9a6b-76dd9e7eeda3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b0bb095424a4e8abc828e0b529d9bc5'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:09:44.910869 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1314/2608] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 04:09:44 2022] DELETE /identity/v3/users/8b0bb095424a4e8abc828e0b529d9bc5 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 04:09:45.154154 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd00e64-8c10-4238-9d37-aa6668208291 service neutron] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:09:45.164527 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd00e64-8c10-4238-9d37-aa6668208291 service neutron] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:09:45.223321 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c11aaa-7782-4320-b8b3-639596f9b939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:09:45.223975 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c11aaa-7782-4320-b8b3-639596f9b939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:09:45.224256 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c11aaa-7782-4320-b8b3-639596f9b939 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:09:45.224524 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c11aaa-7782-4320-b8b3-639596f9b939 admin admin] PATH_INFO: `/v3/projects/8dbea8f697124e6bb449a9ed71670dda` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:09:45.225644 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c11aaa-7782-4320-b8b3-639596f9b939 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dbea8f697124e6bb449a9ed71670dda)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:09:45.227512 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c11aaa-7782-4320-b8b3-639596f9b939 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:09:45.330972 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:09:45.337051 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-b7c11aaa-7782-4320-b8b3-639596f9b939 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dbea8f697124e6bb449a9ed71670dda', 'request_id': 'req-b7c11aaa-7782-4320-b8b3-639596f9b939'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:09:45.376164 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-b7c11aaa-7782-4320-b8b3-639596f9b939 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dbea8f697124e6bb449a9ed71670dda', 'request_id': 'req-b7c11aaa-7782-4320-b8b3-639596f9b939'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:09:45.381618 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-b7c11aaa-7782-4320-b8b3-639596f9b939 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dbea8f697124e6bb449a9ed71670dda', 'request_id': 'req-b7c11aaa-7782-4320-b8b3-639596f9b939'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:09:45.392457 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-b7c11aaa-7782-4320-b8b3-639596f9b939 admin admin] The token cache is being invalidate because project 8dbea8f697124e6bb449a9ed71670dda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 04:09:45.392742 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-b7c11aaa-7782-4320-b8b3-639596f9b939 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:09:45.393361 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-b7c11aaa-7782-4320-b8b3-639596f9b939 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:09:45.394817 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1295/2609] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 04:09:45 2022] DELETE /identity/v3/projects/8dbea8f697124e6bb449a9ed71670dda => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 04:09:45.398757 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a0c11-44d4-4889-9a6b-76dd9e7eeda3 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:09:45.400189 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-879a0c11-44d4-4889-9a6b-76dd9e7eeda3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:09:45.413215 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a0c11-44d4-4889-9a6b-76dd9e7eeda3 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:09:45.467538 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06506d77-2db7-4941-9a42-2dd28b995174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:09:45.468239 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06506d77-2db7-4941-9a42-2dd28b995174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:09:45.468463 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06506d77-2db7-4941-9a42-2dd28b995174 admin admin] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:09:45.468731 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06506d77-2db7-4941-9a42-2dd28b995174 admin admin] PATH_INFO: `/v3/users/099351bc2fbe4f3bb6dc51aa4cf13b13` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:09:45.469845 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06506d77-2db7-4941-9a42-2dd28b995174 admin admin] RBAC: Authorizing `identity:delete_user(user_id=099351bc2fbe4f3bb6dc51aa4cf13b13)` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:09:45.471724 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06506d77-2db7-4941-9a42-2dd28b995174 admin admin] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:09:45.619212 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-06506d77-2db7-4941-9a42-2dd28b995174 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '099351bc2fbe4f3bb6dc51aa4cf13b13'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:09:45.620365 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-06506d77-2db7-4941-9a42-2dd28b995174 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '099351bc2fbe4f3bb6dc51aa4cf13b13'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:09:45.702397 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-06506d77-2db7-4941-9a42-2dd28b995174 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '099351bc2fbe4f3bb6dc51aa4cf13b13'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:09:45.715538 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-06506d77-2db7-4941-9a42-2dd28b995174 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '099351bc2fbe4f3bb6dc51aa4cf13b13'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:09:45.716827 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.notifications [None req-06506d77-2db7-4941-9a42-2dd28b995174 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '099351bc2fbe4f3bb6dc51aa4cf13b13'} {{(pid=88299) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:09:45.719816 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:09:45.729640 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1315/2610] 10.222.0.17 () {58 vars in 1224 bytes} [Mon Oct 17 04:09:45 2022] DELETE /identity/v3/users/099351bc2fbe4f3bb6dc51aa4cf13b13 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 04:09:45.949231 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c11aaa-7782-4320-b8b3-639596f9b939 admin admin] Authenticating user token {{(pid=88300) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:09:45.951342 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.fernet_utils [None req-b7c11aaa-7782-4320-b8b3-639596f9b939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88300) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:09:45.963934 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c11aaa-7782-4320-b8b3-639596f9b939 admin admin] Validating token access rules against request {{(pid=88300) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:09:46.020350 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3033d1-f3c6-4acf-8fae-4c16c8268789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f09f98306cd54b068020428d74bd0957', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f518d7d57a2406a8345926faf8a2f14', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88300) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:09:46.021007 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3033d1-f3c6-4acf-8fae-4c16c8268789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:09:46.021286 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3033d1-f3c6-4acf-8fae-4c16c8268789 admin admin] SCRIPT_NAME: `/identity` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:09:46.021557 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3033d1-f3c6-4acf-8fae-4c16c8268789 admin admin] PATH_INFO: `/v3/projects/a48f898280ab4371a905047720968afa` {{(pid=88300) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:09:46.022654 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3033d1-f3c6-4acf-8fae-4c16c8268789 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a48f898280ab4371a905047720968afa)` {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:09:46.024559 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3033d1-f3c6-4acf-8fae-4c16c8268789 admin admin] RBAC: Authorization granted {{(pid=88300) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:09:46.118686 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-ca3033d1-f3c6-4acf-8fae-4c16c8268789 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a48f898280ab4371a905047720968afa', 'request_id': 'req-ca3033d1-f3c6-4acf-8fae-4c16c8268789'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:09:46.209898 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-ca3033d1-f3c6-4acf-8fae-4c16c8268789 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a48f898280ab4371a905047720968afa', 'request_id': 'req-ca3033d1-f3c6-4acf-8fae-4c16c8268789'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:09:46.216054 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-ca3033d1-f3c6-4acf-8fae-4c16c8268789 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a48f898280ab4371a905047720968afa', 'request_id': 'req-ca3033d1-f3c6-4acf-8fae-4c16c8268789'} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:09:46.227198 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-ca3033d1-f3c6-4acf-8fae-4c16c8268789 admin admin] The token cache is being invalidate because project a48f898280ab4371a905047720968afa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88300) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 17 04:09:46.227419 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-ca3033d1-f3c6-4acf-8fae-4c16c8268789 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:09:46.228046 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG keystone.notifications [None req-ca3033d1-f3c6-4acf-8fae-4c16c8268789 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88300) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 17 04:09:46.229496 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: [pid: 88300|app: 0|req: 1296/2611] 10.222.0.17 () {58 vars in 1233 bytes} [Mon Oct 17 04:09:45 2022] DELETE /identity/v3/projects/a48f898280ab4371a905047720968afa => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 17 04:09:55.810739 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=4 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:09:56.224135 dv-cinder-815582-16-iscsi devstack@keystone.service[88300]: DEBUG dbcounter [-] [88300] Writing DB stats keystone:SELECT=60,keystone:DELETE=11,keystone:INSERT=2 {{(pid=88300) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 17 04:14:13.776803 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06506d77-2db7-4941-9a42-2dd28b995174 admin admin] Authenticating user token {{(pid=88299) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 17 04:14:13.778226 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-06506d77-2db7-4941-9a42-2dd28b995174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:14:13.846339 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06506d77-2db7-4941-9a42-2dd28b995174 admin admin] Validating token access rules against request {{(pid=88299) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 17 04:14:13.902662 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd4cb5-7666-47ab-822e-297943c03260 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b500a113a2aa4a95875d7e094b917db3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '932cdb9ef7644bf5ba57c6d1d6bce7ff', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88299) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 17 04:14:13.903265 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd4cb5-7666-47ab-822e-297943c03260 service placement] REQUEST_METHOD: `GET` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 17 04:14:13.903542 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd4cb5-7666-47ab-822e-297943c03260 service placement] SCRIPT_NAME: `/identity` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 17 04:14:13.903843 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd4cb5-7666-47ab-822e-297943c03260 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88299) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 17 04:14:13.905569 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.fernet_utils [None req-eecd4cb5-7666-47ab-822e-297943c03260 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88299) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 17 04:14:13.916417 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecd4cb5-7666-47ab-822e-297943c03260 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 17 04:14:13.917962 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecd4cb5-7666-47ab-822e-297943c03260 service placement] RBAC: Authorization granted {{(pid=88299) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 17 04:14:13.927521 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: [pid: 88299|app: 0|req: 1316/2612] 10.222.0.17 () {60 vars in 1365 bytes} [Mon Oct 17 04:14:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 17 04:14:23.923440 dv-cinder-815582-16-iscsi devstack@keystone.service[88299]: DEBUG dbcounter [-] [88299] Writing DB stats keystone:SELECT=58 {{(pid=88299) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}